Jul 14 19:38:20.232570 ip-10-0-130-104 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 19:38:23.063729 ip-10-0-130-104 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 19:38:23.063729 ip-10-0-130-104 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 19:38:23.063729 ip-10-0-130-104 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 19:38:23.063729 ip-10-0-130-104 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 19:38:23.063729 ip-10-0-130-104 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 19:38:23.064455 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.064346    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 19:38:23.067888 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067872    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067890    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067895    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067899    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067903    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067906    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067909    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067912    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067915    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067917    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067920    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067922    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067925    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 19:38:23.067921 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067928    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067931    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067934    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067937    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067939    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067948    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067950    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067953    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067956    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067959    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067961    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067964    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067966    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067969    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067973    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067976    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067979    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067983    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067986    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067989    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 19:38:23.068231 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067992    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067995    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.067998    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068001    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068005    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068007    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068010    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068013    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068015    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068018    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068021    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068023    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068026    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068029    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068032    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068035    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068038    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068041    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068044    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 19:38:23.068778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068047    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068049    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068052    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068054    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068057    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068059    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068062    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068064    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068067    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068070    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068072    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068075    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068077    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068079    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068082    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068084    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068087    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068090    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068092    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068095    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 19:38:23.069257 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068097    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068099    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068102    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068104    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068107    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068109    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068111    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068115    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068118    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068120    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068123    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068126    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068129    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068131    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068134    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068137    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068139    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068142    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068144    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068146    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 19:38:23.070018 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068149    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068151    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068154    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068158    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068160    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068162    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.068165    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070218    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070230    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070234    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070237    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070240    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070243    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070246    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070251    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070256    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070259    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070262    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070265    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070268    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 19:38:23.070510 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070271    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070273    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070275    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070278    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070280    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070283    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070286    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070289    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070291    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070294    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070296    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070299    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070301    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070304    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070308    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070311    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070314    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070317    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070319    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070322    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 19:38:23.071004 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070325    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070328    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070331    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070333    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070336    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070339    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070341    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070344    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070346    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070349    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070352    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070354    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070357    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070359    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070362    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070364    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070367    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070369    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070371    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070375    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 19:38:23.071529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070377    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070380    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070382    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070385    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070389    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070393    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070413    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070416    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070419    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070421    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070424    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070427    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070430    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070432    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070435    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070438    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070440    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070443    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070445    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 19:38:23.072046 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070448    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070452    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070456    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070458    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070461    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070464    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070467    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070470    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070473    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070477    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070479    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070482    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070485    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070487    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070490    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070493    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070495    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070498    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070501    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070503    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 19:38:23.072517 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070506    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070509    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070511    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070515    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070517    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070520    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070523    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070911    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070916    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070919    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070921    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070924    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070927    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070930    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070932    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070935    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070938    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070940    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070943    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070946    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 19:38:23.073000 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070949    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070951    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070954    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070958    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070962    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070965    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070968    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070971    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070973    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070976    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070979    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070982    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070984    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070987    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070991    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070995    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.070998    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071001    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071004    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 19:38:23.073529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071007    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071010    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071013    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071016    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071019    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071021    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071025    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071028    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071030    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071033    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071037    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071040    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071043    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071045    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071048    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071050    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071053    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071056    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071058    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071062    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 19:38:23.073980 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071064    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071067    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071071    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071074    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071077    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071080    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071082    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071085    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071088    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071091    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071093    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071096    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071099    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071101    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071104    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071106    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071109    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071111    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071114    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071117    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 19:38:23.074470 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071119    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071122    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071124    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071127    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071129    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071132    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071134    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071138    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071140    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071143    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071146    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071148    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071151    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071153    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071156    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071158    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071161    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071163    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071166    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 19:38:23.074947 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071168    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071171    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071173    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071176    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071178    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071180    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071183    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071185    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071255    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071263    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071269    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071276    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071280    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071284    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071288    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071292    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071296    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071299    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071302    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071305    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071308    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071311    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071314    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 19:38:23.075410 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071317    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071328    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071331    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071336    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071338    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071341    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071344    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071347    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071355    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071358    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071361    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071365    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071368    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071371    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071373    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071377    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071380    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071384    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071387    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071389    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071394    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071410    2581 flags.go:64] FLAG: --enable-server="true"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071413    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071417    2581 flags.go:64] FLAG: --event-burst="100"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071420    2581 flags.go:64] FLAG: --event-qps="50"
Jul 14 19:38:23.075965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071424    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071427    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071429    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071433    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071436    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071439    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071442    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071445    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071448    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071451    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071454    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071457    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071460    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071462    2581 flags.go:64] FLAG: --feature-gates=""
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071466    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071469    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071472    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071475    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071478    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071481    2581 flags.go:64] FLAG: --help="false"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071484    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071488    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071491    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 19:38:23.076560 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071494    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071497    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071500    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071503    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071508    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071512    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071515    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071518    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071521    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071524    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071527    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071530    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071533    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071536    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071539    2581 flags.go:64] FLAG: --lock-file=""
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071543    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071545    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071548    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071554    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071557    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071560    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071563    2581 flags.go:64] FLAG: --logging-format="text"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071566    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071569    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 19:38:23.077135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071572    2581 flags.go:64] FLAG: --manifest-url=""
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071575    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071579    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071583    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071586    2581 flags.go:64] FLAG: --max-pods="110"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071589    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071592    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071595    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071598    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071601    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071604    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071607    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071616    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071620    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071623    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071626    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071629    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071634    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071637    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071640    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071643    2581 flags.go:64] FLAG: --port="10250"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071646    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071650    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b584780a0c40c364"
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071653    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 14 19:38:23.077828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071656    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071659    2581 flags.go:64] FLAG: --register-node="true"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071662    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071665    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071669    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071672    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071674    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071678    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071681    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071684    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071687    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071689    2581 flags.go:64] FLAG: --runonce="false"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071692    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071695    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071698    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071701    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071704    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071707    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071710    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071714    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071718    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071721    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071726    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071729    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071732    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 19:38:23.078421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071735    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071738    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071743    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071746    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071749    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071753    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071756    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071759    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071762    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071765    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071768    2581 flags.go:64] FLAG: --v="2"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071772    2581 flags.go:64] FLAG: --version="false"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071776    2581 flags.go:64] FLAG: --vmodule=""
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071779    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.071783    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071902    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071906    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071909    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071912    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071914    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071917    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071920    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071922    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 19:38:23.079021 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071925    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071928    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071930    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071933    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071937    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071940    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071944    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071946    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071949    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071952    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071954    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071957    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071959    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071962    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071965    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071968    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071971    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071974    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071976    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071979    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 19:38:23.079585 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071982    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071984    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071988    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071990    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071993    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071995    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.071998    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072000    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072003    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072005    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072007    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072010    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072012    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072015    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072017    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072020    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072024    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072027    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072032    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072034    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 19:38:23.080108 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072037    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072040    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072042    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072045    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072048    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072051    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072053    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072056    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072059    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072061    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072064    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072067    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072069    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072071    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072074    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072081    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072084    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072086    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072089    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072091    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 19:38:23.080618 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072093    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072096    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072099    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072102    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072104    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072106    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072109    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072113    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072118    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072121    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072126    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072128    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072131    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072133    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072135    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072138    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072141    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072144    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072146    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 19:38:23.081093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072149    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072151    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072154    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072156    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072159    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072161    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072164    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072166    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072169    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072177    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072180    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.072184    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.072191    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.079112    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.079133    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 19:38:23.081563 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.079754    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.079766    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079828    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079834    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079838    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079841    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079844    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079847    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079850    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079853    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079856    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079859    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079862    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079864    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079867    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079870    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079873    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079875    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079878    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 19:38:23.081949 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079880    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079883    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079886    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079889    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079892    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079895    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079898    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079900    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079903    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079905    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079908    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079910    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079915    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079918    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079920    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079923    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079925    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079928    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079930    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079933    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 19:38:23.082452 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079935    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079938    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079940    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079943    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079945    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079948    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079951    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079954    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079957    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079960    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079962    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079965    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079967    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079970    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079973    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079975    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079978    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079981    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079983    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079986    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 19:38:23.082945 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079988    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079991    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079993    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.079998    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080002    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080006    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080009    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080012    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080016    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080019    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080022    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080024    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080027    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080030    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080033    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080036    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080039    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080042    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080045    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 19:38:23.083460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080047    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080051    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080055    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080059    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080063    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080066    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080068    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080071    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080073    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080076    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080078    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080081    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080083    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080086    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080089    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080092    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080095    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080098    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080101    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 19:38:23.083948 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080104    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080107    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080110    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080113    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.080118    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080259    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080264    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080268    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080271    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080274    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080277    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080281    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080285    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080288    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 19:38:23.084392 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080291    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080294    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080297    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080301    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080303    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080306    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080309    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080311    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080313    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080316    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080319    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080323    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080327    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080330    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080332    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080335    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080338    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080341    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080343    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080346    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 19:38:23.085017 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080349    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080352    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080354    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080357    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080359    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080362    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080365    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080367    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080370    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080372    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080375    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080377    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080379    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080382    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080385    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080387    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080390    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080392    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080411    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 19:38:23.085529 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080416    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080419    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080421    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080424    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080427    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080430    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080433    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080436    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080438    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080441    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080444    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080447    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080449    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080453    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080456    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080459    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080461    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080464    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080467    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080469    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 19:38:23.086010 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080472    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080474    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080477    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080481    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080484    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080487    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080490    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080493    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080496    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080499    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080502    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080505    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080507    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080510    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080512    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080515    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080518    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080522    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080525    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 19:38:23.086757 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080527    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080530    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080533    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080536    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080538    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080541    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080543    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080546    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080549    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080552    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080554    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:23.080556    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.080561    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.080695    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.083296    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 19:38:23.087256 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.084559    2581 server.go:1019] "Starting client certificate rotation"
Jul 14 19:38:23.087820 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.084679    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 19:38:23.087820 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.084723    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 19:38:23.087820 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.086666    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 19:38:23.087930 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.087866    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 19:38:23.097359 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.097335    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 14 19:38:23.100262 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.100242    2581 log.go:25] "Validated CRI v1 image API"
Jul 14 19:38:23.102589 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.102486    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 19:38:23.109025 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.108997    2581 fs.go:135] Filesystem UUIDs: map[4a96f139-2632-443a-847c-e55c140f149f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a120f389-fcc7-40c8-ae48-bc23a772d74f:/dev/nvme0n1p4]
Jul 14 19:38:23.109141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.109020    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 19:38:23.114809 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.114679    2581 manager.go:216] Machine: {Timestamp:2026-07-14 19:38:23.113502665 +0000 UTC m=+1.425202112 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104029 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24fb0740c9fadfd3b97f6109371dc7 SystemUUID:ec24fb07-40c9-fadf-d3b9-7f6109371dc7 BootID:9977f786-64f5-4f49-b867-08e52677c422 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4b:03:08:52:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4b:03:08:52:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:b2:78:56:99:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 19:38:23.114809 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.114802    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 19:38:23.114944 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.114906    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 19:38:23.116465 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.116422    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 19:38:23.116651 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.116467    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 19:38:23.117235 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.117224    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 19:38:23.117266 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.117237    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 19:38:23.117266 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.117245    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 19:38:23.117266 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.117264    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 19:38:23.118535 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.118523    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 19:38:23.122234 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.122217    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 19:38:23.122392 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.122382    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 19:38:23.123002 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.122991    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 19:38:23.123034 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.123009    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 19:38:23.123034 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.123030    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 19:38:23.123091 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.123042    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 14 19:38:23.123091 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.123055    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 19:38:23.123585 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.123570    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 19:38:23.123639 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.123598    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 19:38:23.125745 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.125726    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 19:38:23.127895 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.127879    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 19:38:23.127949 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.127911    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 19:38:23.128085 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128075    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 19:38:23.128122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128091    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 19:38:23.128122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128097    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 19:38:23.128122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128103    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 19:38:23.128122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128111    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 19:38:23.128122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128117    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 19:38:23.128279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128202    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 19:38:23.128540 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128529    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 19:38:23.128540 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128540    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 19:38:23.128601 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128547    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 19:38:23.128601 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128552    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 19:38:23.128601 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128562    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 19:38:23.128601 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.128571    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 19:38:23.129239 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.129230    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 19:38:23.129267 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.129239    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 19:38:23.132581 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.132566    2581 server.go:1268] "Started kubelet"
Jul 14 19:38:23.132683 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.132642    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 19:38:23.132732 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.132658    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 19:38:23.132776 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.132752    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 19:38:23.133492 ip-10-0-130-104 systemd[1]: Started Kubernetes Kubelet.
Jul 14 19:38:23.134801 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.134783    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 19:38:23.136096 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.136081    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 19:38:23.138634 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.138612    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 19:38:23.139084 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.139059    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 19:38:23.141627 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.138961    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35e6f16638  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.132534328 +0000 UTC m=+1.444233775,LastTimestamp:2026-07-14 19:38:23.132534328 +0000 UTC m=+1.444233775,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.141825 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.141792    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 19:38:23.142360 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.142339    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 19:38:23.144619 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.144598    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 19:38:23.144727 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.144707    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.144816 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.144601    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 19:38:23.145223 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145208    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 19:38:23.145323 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145311    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 19:38:23.145483 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145209    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 19:38:23.145589 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145569    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 19:38:23.145589 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145583    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 19:38:23.145697 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145666    2581 factory.go:55] Registering systemd factory
Jul 14 19:38:23.145697 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145684    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 14 19:38:23.145927 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145911    2581 factory.go:153] Registering CRI-O factory
Jul 14 19:38:23.145990 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145929    2581 factory.go:223] Registration of the crio container factory successfully
Jul 14 19:38:23.145990 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.145981    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 19:38:23.146090 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.146007    2581 factory.go:103] Registering Raw factory
Jul 14 19:38:23.146090 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.146023    2581 manager.go:1196] Started watching for new ooms in manager
Jul 14 19:38:23.146950 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.146930    2581 manager.go:349] Starting recovery of all containers
Jul 14 19:38:23.157246 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.157189    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 19:38:23.157416 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.157278    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 19:38:23.157626 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.157609    2581 manager.go:354] Recovery completed
Jul 14 19:38:23.162483 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.162467    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:23.164797 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.164762    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:23.164856 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.164824    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:23.164856 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.164842    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:23.165937 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.165920    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 19:38:23.165937 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.165933    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 19:38:23.166044 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.165954    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 19:38:23.166044 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.165954    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:23.166789 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.166771    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:23.166867 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.166806    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:23.166867 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.166821    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:23.168069 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.168055    2581 policy_none.go:49] "None policy: Start"
Jul 14 19:38:23.168158 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.168073    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 19:38:23.168158 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.168084    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 19:38:23.170089 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.170076    2581 policy_none.go:47] "Start"
Jul 14 19:38:23.179006 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.178934    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35e8ddbaa9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.164799657 +0000 UTC m=+1.476499107,LastTimestamp:2026-07-14 19:38:23.164799657 +0000 UTC m=+1.476499107,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.183502 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.183433    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35e8de37b0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-104.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.164831664 +0000 UTC m=+1.476531112,LastTimestamp:2026-07-14 19:38:23.164831664 +0000 UTC m=+1.476531112,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.199956 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.199882    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35e8de7aea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-104.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.164848874 +0000 UTC m=+1.476548322,LastTimestamp:2026-07-14 19:38:23.164848874 +0000 UTC m=+1.476548322,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.211876 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.211785    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-104.ec2.internal.18c23f35e8ddbaa9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35e8ddbaa9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.164799657 +0000 UTC m=+1.476499107,LastTimestamp:2026-07-14 19:38:23.16678772 +0000 UTC m=+1.478487172,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.219140 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.219120    2581 manager.go:337] "Starting Device Plugin manager"
Jul 14 19:38:23.219258 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.219163    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 19:38:23.219258 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.219175    2581 server.go:87] "Starting device plugin registration server"
Jul 14 19:38:23.219496 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.219483    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 19:38:23.219552 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.219496    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 19:38:23.219618 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.219586    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 19:38:23.219688 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.219676    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 19:38:23.219740 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.219689    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 19:38:23.221231 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.220586    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 19:38:23.221231 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.220642    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.224878 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.224804    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-104.ec2.internal.18c23f35e8de37b0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35e8de37b0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-104.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.164831664 +0000 UTC m=+1.476531112,LastTimestamp:2026-07-14 19:38:23.166812866 +0000 UTC m=+1.478512315,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.236155 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.236059    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-104.ec2.internal.18c23f35e8de7aea\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35e8de7aea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-104.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.164848874 +0000 UTC m=+1.476548322,LastTimestamp:2026-07-14 19:38:23.166829351 +0000 UTC m=+1.478528807,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.248051 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.247958    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.ec2.internal.18c23f35ec482894  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.ec2.internal,UID:ip-10-0-130-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.ec2.internal,},FirstTimestamp:2026-07-14 19:38:23.22210626 +0000 UTC m=+1.533805695,LastTimestamp:2026-07-14 19:38:23.22210626 +0000 UTC m=+1.533805695,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.ec2.internal,}"
Jul 14 19:38:23.282776 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.282751    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hrwvz"
Jul 14 19:38:23.305036 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.305005    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hrwvz"
Jul 14 19:38:23.320046 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.319978    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:23.320986 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.320971    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:23.321037 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.321004    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:23.321037 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.321016    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:23.321110 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.321042    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.330019 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.329971    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 19:38:23.331285 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.331267    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 19:38:23.331385 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.331315    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 19:38:23.331385 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.331339    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 19:38:23.331385 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.331379    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 19:38:23.333569 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.333549    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 19:38:23.336502 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.336482    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.336592 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.336508    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-104.ec2.internal\": node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.378929 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.378897    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.432418 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.432356    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"]
Jul 14 19:38:23.432592 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.432475    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:23.433489 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.433470    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:23.433627 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.433506    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:23.433627 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.433521    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:23.435726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.435712    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:23.435841 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.435824    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.435876 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.435865    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:23.436549 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.436532    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:23.436627 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.436566    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:23.436627 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.436579    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:23.436627 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.436535    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:23.436719 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.436633    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:23.436719 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.436648    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:23.438653 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.438634    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.438760 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.438665    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:23.439476 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.439449    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:23.439556 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.439482    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:23.439556 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.439493    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:23.446266 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.446241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7a9b0255c9ea7786828afe16280e2bd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.ec2.internal\" (UID: \"b7a9b0255c9ea7786828afe16280e2bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.446352 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.446273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/833569dec28ad50c40c5372b18ab9624-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal\" (UID: \"833569dec28ad50c40c5372b18ab9624\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.446352 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.446292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/833569dec28ad50c40c5372b18ab9624-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal\" (UID: \"833569dec28ad50c40c5372b18ab9624\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.459352 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.459328    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-104.ec2.internal\" not found" node="ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.463843 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.463821    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-104.ec2.internal\" not found" node="ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.479918 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.479879    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.546417 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.546378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7a9b0255c9ea7786828afe16280e2bd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.ec2.internal\" (UID: \"b7a9b0255c9ea7786828afe16280e2bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.546546 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.546451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/833569dec28ad50c40c5372b18ab9624-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal\" (UID: \"833569dec28ad50c40c5372b18ab9624\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.546546 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.546477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/833569dec28ad50c40c5372b18ab9624-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal\" (UID: \"833569dec28ad50c40c5372b18ab9624\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.546546 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.546504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7a9b0255c9ea7786828afe16280e2bd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.ec2.internal\" (UID: \"b7a9b0255c9ea7786828afe16280e2bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.546546 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.546524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/833569dec28ad50c40c5372b18ab9624-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal\" (UID: \"833569dec28ad50c40c5372b18ab9624\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.546669 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.546528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/833569dec28ad50c40c5372b18ab9624-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal\" (UID: \"833569dec28ad50c40c5372b18ab9624\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.581010 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.580932    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.681513 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.681476    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.760941 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.760912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.766826 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:23.766700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:23.782483 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.782454    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.883060 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.882954    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:23.983563 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:23.983517    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:24.036633 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.036599    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 19:38:24.083696 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:24.083657    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:24.083696 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.083666    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 19:38:24.084214 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.083837    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 19:38:24.084214 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.083877    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 19:38:24.184642 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:24.184538    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:24.251163 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.251127    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 19:38:24.252193 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.252173    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 19:38:24.252273 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.252207    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 19:38:24.252273 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.252217    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeHasSufficientPID"
Jul 14 19:38:24.284890 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:24.284850    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:24.309021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.308987    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 19:33:23 +0000 UTC" deadline="2028-01-18 16:52:55.311571886 +0000 UTC"
Jul 14 19:38:24.309021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.309021    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13269h14m31.002554979s"
Jul 14 19:38:24.385980 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:24.385933    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:24.414910 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:24.414870    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a9b0255c9ea7786828afe16280e2bd.slice/crio-4760073c492e5cb8e0b565190a72949abd46066ad859a14a7c85c347e9b1498d WatchSource:0}: Error finding container 4760073c492e5cb8e0b565190a72949abd46066ad859a14a7c85c347e9b1498d: Status 404 returned error can't find the container with id 4760073c492e5cb8e0b565190a72949abd46066ad859a14a7c85c347e9b1498d
Jul 14 19:38:24.419537 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.419518    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 19:38:24.486560 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:24.486475    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:24.587043 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:24.586995    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-104.ec2.internal\" not found"
Jul 14 19:38:24.607576 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.607547    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 19:38:24.645608 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.645575    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:24.655362 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.655338    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 19:38:24.656141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.656124    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal"
Jul 14 19:38:24.660073 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.660049    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 19:38:24.680702 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.680674    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 19:38:24.998592 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:24.998559    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 19:38:25.125087 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.125057    2581 apiserver.go:52] "Watching apiserver"
Jul 14 19:38:25.132692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.132650    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 19:38:25.133115 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.133077    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zdsp5","kube-system/konnectivity-agent-n99s2","kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc","openshift-cluster-node-tuning-operator/tuned-9jdvp","openshift-image-registry/node-ca-5p8x5","openshift-multus/multus-additional-cni-plugins-v2sss","openshift-network-operator/iptables-alerter-lqhxq","openshift-dns/node-resolver-z826t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal","openshift-multus/multus-pbdmt","openshift-multus/network-metrics-daemon-4jvmg","openshift-network-diagnostics/network-check-target-49nh2","openshift-ovn-kubernetes/ovnkube-node-pmrvf"]
Jul 14 19:38:25.135787 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.135761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.138146 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.138117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:25.138271 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.138213    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:25.138271 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.138230    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 19:38:25.138353 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.138295    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 19:38:25.138735 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.138517    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 19:38:25.138735 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.138588    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 19:38:25.138735 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.138633    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9f8vd\""
Jul 14 19:38:25.138951 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.138780    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 19:38:25.140557 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.140513    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.143261 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.143237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.143448 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.143344    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 19:38:25.143565 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.143534    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 19:38:25.143734 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.143715    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 19:38:25.144149 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.144129    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 19:38:25.144765 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.144470    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 19:38:25.144765 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.144593    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 19:38:25.144765 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.144681    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zvq7f\""
Jul 14 19:38:25.145646 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.145626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.145942 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.145927    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 19:38:25.146087 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.146071    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 19:38:25.146921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.146739    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 19:38:25.146921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.146797    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rhdc7\""
Jul 14 19:38:25.147261 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.147238    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 19:38:25.147921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.147903    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.148018 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.147948    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 19:38:25.148262 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.148246    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 19:38:25.148796 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.148777    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9s2gl\""
Jul 14 19:38:25.150529 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.150511    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.150852 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.150657    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 19:38:25.150852 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.150779    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 19:38:25.151027 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.151008    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-47hc6\""
Jul 14 19:38:25.151119 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.151088    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 19:38:25.152864 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.152845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.152962 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.152927    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:25.154120 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.154099    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5sk5g\""
Jul 14 19:38:25.154324 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.154310    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 19:38:25.154394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.154377    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 19:38:25.155360 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.155754 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-var-lib-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.155838 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-systemd\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.155838 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-tuned\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.155838 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4250805b-0e7c-4d7d-8095-09ea841b2fa7-tmp\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.155990 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-log-socket\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.155990 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.155990 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.155974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.156141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovnkube-script-lib\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.156141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysconfig\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.156141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-sys\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.156141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13c49190-4865-4f90-a7f6-3400303c9879-host\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.156141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156096    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-ovn\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.156141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-node-log\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovn-node-metrics-cert\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-etc-selinux\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysctl-d\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysctl-conf\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-cni-netd\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-os-release\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvnqd\" (UniqueName: \"kubernetes.io/projected/0777416d-4547-40b1-82f8-78911a02ad4a-kube-api-access-wvnqd\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.156444 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-socket-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-device-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-sys-fs\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-system-cni-dir\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-systemd-units\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156698    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsv7x\" (UniqueName: \"kubernetes.io/projected/a578d7c0-b23f-4285-984c-d1a2c313d167-kube-api-access-rsv7x\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-kubernetes\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-var-lib-kubelet\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13c49190-4865-4f90-a7f6-3400303c9879-serviceca\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q97v6\" (UniqueName: \"kubernetes.io/projected/13c49190-4865-4f90-a7f6-3400303c9879-kube-api-access-q97v6\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156875    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.156897 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-cni-bin\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovnkube-config\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156968    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-lib-modules\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.156997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-host\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrc9f\" (UniqueName: \"kubernetes.io/projected/4250805b-0e7c-4d7d-8095-09ea841b2fa7-kube-api-access-xrc9f\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-cnibin\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-env-overrides\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-registration-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-modprobe-d\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-run\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-systemd\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-etc-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-kubelet\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-slash\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-run-netns\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.157394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk7kh\" (UniqueName: \"kubernetes.io/projected/92ada39f-8715-4093-b7ff-5db5af59b6b0-kube-api-access-pk7kh\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.158117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.158117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.158117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:25.158117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157592    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 19:38:25.158117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.158117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157924    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nqncm\""
Jul 14 19:38:25.158117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.157961    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 19:38:25.158324 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.158256    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 19:38:25.159858 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.159836    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 19:38:25.161151 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.161127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.162237 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.161826    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 19:38:25.162237 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.162010    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xnpcv\""
Jul 14 19:38:25.162735 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.162488    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 19:38:25.164096 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.164076    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 19:38:25.164219 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.164146    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9qk8w\""
Jul 14 19:38:25.165416 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.165383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:25.165524 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.165498    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:25.190391 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.190368    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fpjlj"
Jul 14 19:38:25.200144 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.200119    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fpjlj"
Jul 14 19:38:25.246696 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.246656    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 19:38:25.258494 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-lib-modules\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.258494 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-host\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.258494 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrc9f\" (UniqueName: \"kubernetes.io/projected/4250805b-0e7c-4d7d-8095-09ea841b2fa7-kube-api-access-xrc9f\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.258494 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-conf-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.258494 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-cnibin\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.258494 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-modprobe-d\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-lib-modules\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258526    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-cni-binary-copy\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-tuned\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4250805b-0e7c-4d7d-8095-09ea841b2fa7-tmp\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-host\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-systemd\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-kubelet\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-slash\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-run-netns\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-var-lib-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-system-cni-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-netns\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258925    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-cni-bin\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-hostroot\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.258994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-cnibin\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259001    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-kubelet\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-modprobe-d\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzjhn\" (UniqueName: \"kubernetes.io/projected/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-kube-api-access-kzjhn\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysconfig\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13c49190-4865-4f90-a7f6-3400303c9879-host\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259305    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a83de8b-07d6-4adc-8f13-d8984c449f1a-host-slash\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259420    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-socket-dir-parent\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-k8s-cni-cncf-io\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovn-node-metrics-cert\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysctl-d\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259546    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysctl-conf\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.259984 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259639    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-var-lib-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysconfig\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-run-netns\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259732    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13c49190-4865-4f90-a7f6-3400303c9879-host\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-cni-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-systemd\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-cni-multus\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-slash\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259879    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysctl-conf\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-device-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsv7x\" (UniqueName: \"kubernetes.io/projected/a578d7c0-b23f-4285-984c-d1a2c313d167-kube-api-access-rsv7x\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259954    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-device-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.259990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-kubernetes\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260031    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-sysctl-d\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28pzt\" (UniqueName: \"kubernetes.io/projected/cc6571ea-7b0d-4671-a89b-e427c40cecc4-kube-api-access-28pzt\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.260893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-kubernetes\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-os-release\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovnkube-config\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec-agent-certs\") pod \"konnectivity-agent-n99s2\" (UID: \"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec\") " pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-env-overrides\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-registration-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-run\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-kubelet\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-etc-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-registration-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260791    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovnkube-config\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-run\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.260976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-etc-openvswitch\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk7kh\" (UniqueName: \"kubernetes.io/projected/92ada39f-8715-4093-b7ff-5db5af59b6b0-kube-api-access-pk7kh\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-env-overrides\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.261811 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a83de8b-07d6-4adc-8f13-d8984c449f1a-iptables-alerter-script\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r2dm\" (UniqueName: \"kubernetes.io/projected/7a83de8b-07d6-4adc-8f13-d8984c449f1a-kube-api-access-9r2dm\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec-konnectivity-ca\") pod \"konnectivity-agent-n99s2\" (UID: \"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec\") " pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-cnibin\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-etc-kubernetes\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261603    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-systemd\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc6571ea-7b0d-4671-a89b-e427c40cecc4-hosts-file\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-dbus\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-multus-certs\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-log-socket\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovnkube-script-lib\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-log-socket\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-sys\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:25.262665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-ovn\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-node-log\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.261975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-etc-selinux\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6571ea-7b0d-4671-a89b-e427c40cecc4-tmp-dir\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4zft\" (UniqueName: \"kubernetes.io/projected/24cb2861-5df6-4718-9b15-38f935fa2397-kube-api-access-b4zft\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-cni-netd\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262111    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-os-release\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvnqd\" (UniqueName: \"kubernetes.io/projected/0777416d-4547-40b1-82f8-78911a02ad4a-kube-api-access-wvnqd\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-socket-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-sys-fs\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-node-log\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-run-ovn\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262364    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-sys\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-var-lib-kubelet\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.263395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-system-cni-dir\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-systemd-units\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13c49190-4865-4f90-a7f6-3400303c9879-serviceca\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q97v6\" (UniqueName: \"kubernetes.io/projected/13c49190-4865-4f90-a7f6-3400303c9879-kube-api-access-q97v6\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-kubelet-config\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-daemon-config\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262605    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-cni-bin\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-cni-bin\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-socket-dir\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-sys-fs\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0777416d-4547-40b1-82f8-78911a02ad4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262845    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a578d7c0-b23f-4285-984c-d1a2c313d167-etc-selinux\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-system-cni-dir\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-systemd-units\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.262988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92ada39f-8715-4093-b7ff-5db5af59b6b0-host-cni-netd\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-var-lib-kubelet\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.264123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0777416d-4547-40b1-82f8-78911a02ad4a-os-release\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.264726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-systemd\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.264726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovnkube-script-lib\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.264726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13c49190-4865-4f90-a7f6-3400303c9879-serviceca\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.264726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92ada39f-8715-4093-b7ff-5db5af59b6b0-ovn-node-metrics-cert\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.264726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263371    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4250805b-0e7c-4d7d-8095-09ea841b2fa7-tmp\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.264726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.263601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4250805b-0e7c-4d7d-8095-09ea841b2fa7-etc-tuned\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.266626 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.266605    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 19:38:25.266748 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.266631    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 19:38:25.266748 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.266645    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zxxtv for pod openshift-network-diagnostics/network-check-target-49nh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:25.266748 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.266708    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv podName:8a29848a-06f0-42fb-96a6-4726020ea757 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:25.766688075 +0000 UTC m=+4.078387511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zxxtv" (UniqueName: "kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv") pod "network-check-target-49nh2" (UID: "8a29848a-06f0-42fb-96a6-4726020ea757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:25.270162 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.269887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsv7x\" (UniqueName: \"kubernetes.io/projected/a578d7c0-b23f-4285-984c-d1a2c313d167-kube-api-access-rsv7x\") pod \"aws-ebs-csi-driver-node-wdcvc\" (UID: \"a578d7c0-b23f-4285-984c-d1a2c313d167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.270429 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.270362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk7kh\" (UniqueName: \"kubernetes.io/projected/92ada39f-8715-4093-b7ff-5db5af59b6b0-kube-api-access-pk7kh\") pod \"ovnkube-node-pmrvf\" (UID: \"92ada39f-8715-4093-b7ff-5db5af59b6b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.270855 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.270769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrc9f\" (UniqueName: \"kubernetes.io/projected/4250805b-0e7c-4d7d-8095-09ea841b2fa7-kube-api-access-xrc9f\") pod \"tuned-9jdvp\" (UID: \"4250805b-0e7c-4d7d-8095-09ea841b2fa7\") " pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.271058 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.271038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvnqd\" (UniqueName: \"kubernetes.io/projected/0777416d-4547-40b1-82f8-78911a02ad4a-kube-api-access-wvnqd\") pod \"multus-additional-cni-plugins-v2sss\" (UID: \"0777416d-4547-40b1-82f8-78911a02ad4a\") " pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.272078 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.272055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q97v6\" (UniqueName: \"kubernetes.io/projected/13c49190-4865-4f90-a7f6-3400303c9879-kube-api-access-q97v6\") pod \"node-ca-5p8x5\" (UID: \"13c49190-4865-4f90-a7f6-3400303c9879\") " pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.336693 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.336635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal" event={"ID":"b7a9b0255c9ea7786828afe16280e2bd","Type":"ContainerStarted","Data":"4760073c492e5cb8e0b565190a72949abd46066ad859a14a7c85c347e9b1498d"}
Jul 14 19:38:25.363586 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec-agent-certs\") pod \"konnectivity-agent-n99s2\" (UID: \"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec\") " pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.363757 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-kubelet\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.363757 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a83de8b-07d6-4adc-8f13-d8984c449f1a-iptables-alerter-script\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.363757 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r2dm\" (UniqueName: \"kubernetes.io/projected/7a83de8b-07d6-4adc-8f13-d8984c449f1a-kube-api-access-9r2dm\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.363757 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363681    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec-konnectivity-ca\") pod \"konnectivity-agent-n99s2\" (UID: \"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec\") " pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.363757 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-kubelet\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-cnibin\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-etc-kubernetes\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc6571ea-7b0d-4671-a89b-e427c40cecc4-hosts-file\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-dbus\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-multus-certs\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363943    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6571ea-7b0d-4671-a89b-e427c40cecc4-tmp-dir\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.363972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4zft\" (UniqueName: \"kubernetes.io/projected/24cb2861-5df6-4718-9b15-38f935fa2397-kube-api-access-b4zft\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-kubelet-config\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.364033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-daemon-config\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364042    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc6571ea-7b0d-4671-a89b-e427c40cecc4-hosts-file\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-conf-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-cni-binary-copy\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-cnibin\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-system-cni-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-etc-kubernetes\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-netns\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-kubelet-config\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-cni-bin\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-hostroot\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364237    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-cni-bin\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzjhn\" (UniqueName: \"kubernetes.io/projected/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-kube-api-access-kzjhn\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a83de8b-07d6-4adc-8f13-d8984c449f1a-host-slash\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364288    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-dbus\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec-konnectivity-ca\") pod \"konnectivity-agent-n99s2\" (UID: \"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec\") " pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-socket-dir-parent\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.364526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-multus-certs\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-k8s-cni-cncf-io\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-cni-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364434    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6571ea-7b0d-4671-a89b-e427c40cecc4-tmp-dir\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.364452    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-k8s-cni-cncf-io\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364379    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-socket-dir-parent\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.364520    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:25.86450027 +0000 UTC m=+4.176199724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-cni-multus\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28pzt\" (UniqueName: \"kubernetes.io/projected/cc6571ea-7b0d-4671-a89b-e427c40cecc4-kube-api-access-28pzt\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-os-release\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364592    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-hostroot\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a83de8b-07d6-4adc-8f13-d8984c449f1a-host-slash\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364657    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-var-lib-cni-multus\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364664    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-conf-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364560    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-cni-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-system-cni-dir\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-os-release\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.365427 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-host-run-netns\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.366282 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-multus-daemon-config\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.366282 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.364884    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a83de8b-07d6-4adc-8f13-d8984c449f1a-iptables-alerter-script\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.366282 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.364949    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:25.366282 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.365012    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret podName:c7a2b6a9-bd74-458a-88a3-b7e5570767d4 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:25.864995444 +0000 UTC m=+4.176694892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret") pod "global-pull-secret-syncer-zdsp5" (UID: "c7a2b6a9-bd74-458a-88a3-b7e5570767d4") : object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:25.366282 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.365257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-cni-binary-copy\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.366629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.366599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec-agent-certs\") pod \"konnectivity-agent-n99s2\" (UID: \"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec\") " pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.376904 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.376865    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzjhn\" (UniqueName: \"kubernetes.io/projected/c8e5430b-eadb-496c-b562-d9c3d6d58e6d-kube-api-access-kzjhn\") pod \"multus-pbdmt\" (UID: \"c8e5430b-eadb-496c-b562-d9c3d6d58e6d\") " pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.376904 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.376898    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4zft\" (UniqueName: \"kubernetes.io/projected/24cb2861-5df6-4718-9b15-38f935fa2397-kube-api-access-b4zft\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:25.377085 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.376922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28pzt\" (UniqueName: \"kubernetes.io/projected/cc6571ea-7b0d-4671-a89b-e427c40cecc4-kube-api-access-28pzt\") pod \"node-resolver-z826t\" (UID: \"cc6571ea-7b0d-4671-a89b-e427c40cecc4\") " pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.377265 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.377249    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r2dm\" (UniqueName: \"kubernetes.io/projected/7a83de8b-07d6-4adc-8f13-d8984c449f1a-kube-api-access-9r2dm\") pod \"iptables-alerter-lqhxq\" (UID: \"7a83de8b-07d6-4adc-8f13-d8984c449f1a\") " pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.448551 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.448511    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v2sss"
Jul 14 19:38:25.456341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.456313    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:25.460208 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:25.460178    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0777416d_4547_40b1_82f8_78911a02ad4a.slice/crio-a42db19b8733cd66bb36e7c78e2e152a402d0cfb67184bb73d55e5cda8384bde WatchSource:0}: Error finding container a42db19b8733cd66bb36e7c78e2e152a402d0cfb67184bb73d55e5cda8384bde: Status 404 returned error can't find the container with id a42db19b8733cd66bb36e7c78e2e152a402d0cfb67184bb73d55e5cda8384bde
Jul 14 19:38:25.465606 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.465577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc"
Jul 14 19:38:25.472350 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.472323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jdvp"
Jul 14 19:38:25.478505 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.478412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5p8x5"
Jul 14 19:38:25.486461 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.486431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z826t"
Jul 14 19:38:25.493271 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.493246    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:25.500953 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.500921    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqhxq"
Jul 14 19:38:25.506777 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.506753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pbdmt"
Jul 14 19:38:25.767576 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.767478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:25.767737 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.767668    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 19:38:25.767737 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.767689    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 19:38:25.767737 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.767702    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zxxtv for pod openshift-network-diagnostics/network-check-target-49nh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:25.767867 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.767767    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv podName:8a29848a-06f0-42fb-96a6-4726020ea757 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:26.767750642 +0000 UTC m=+5.079450079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxxtv" (UniqueName: "kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv") pod "network-check-target-49nh2" (UID: "8a29848a-06f0-42fb-96a6-4726020ea757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:25.868159 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.868124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:25.868328 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:25.868212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:25.868328 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.868266    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:25.868328 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.868304    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:25.868476 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.868339    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:26.868320844 +0000 UTC m=+5.180020302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:25.868476 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:25.868354    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret podName:c7a2b6a9-bd74-458a-88a3-b7e5570767d4 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:26.86834768 +0000 UTC m=+5.180047118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret") pod "global-pull-secret-syncer-zdsp5" (UID: "c7a2b6a9-bd74-458a-88a3-b7e5570767d4") : object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:26.115680 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.115592    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ada39f_8715_4093_b7ff_5db5af59b6b0.slice/crio-9d1e57a6a74fae47b6d7df17c115c0ff7a79a1c304e07b782234933903268994 WatchSource:0}: Error finding container 9d1e57a6a74fae47b6d7df17c115c0ff7a79a1c304e07b782234933903268994: Status 404 returned error can't find the container with id 9d1e57a6a74fae47b6d7df17c115c0ff7a79a1c304e07b782234933903268994
Jul 14 19:38:26.128211 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.128124    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4250805b_0e7c_4d7d_8095_09ea841b2fa7.slice/crio-275e209fda1fec315a17545b1a19ab36b94dce5b4f02c7913d921bf9cbeb101e WatchSource:0}: Error finding container 275e209fda1fec315a17545b1a19ab36b94dce5b4f02c7913d921bf9cbeb101e: Status 404 returned error can't find the container with id 275e209fda1fec315a17545b1a19ab36b94dce5b4f02c7913d921bf9cbeb101e
Jul 14 19:38:26.130694 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.130607    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e5430b_eadb_496c_b562_d9c3d6d58e6d.slice/crio-2fc755fe87c068a0ebd90a159f3dcd1eda1d14dcdad859d61f43b689edaf8538 WatchSource:0}: Error finding container 2fc755fe87c068a0ebd90a159f3dcd1eda1d14dcdad859d61f43b689edaf8538: Status 404 returned error can't find the container with id 2fc755fe87c068a0ebd90a159f3dcd1eda1d14dcdad859d61f43b689edaf8538
Jul 14 19:38:26.133671 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.133631    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c49190_4865_4f90_a7f6_3400303c9879.slice/crio-e66df771c0dc3e6d3a9221453d181010291847215c2782d725824b90ab2996c8 WatchSource:0}: Error finding container e66df771c0dc3e6d3a9221453d181010291847215c2782d725824b90ab2996c8: Status 404 returned error can't find the container with id e66df771c0dc3e6d3a9221453d181010291847215c2782d725824b90ab2996c8
Jul 14 19:38:26.134236 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.134209    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda578d7c0_b23f_4285_984c_d1a2c313d167.slice/crio-ce84963b65abc1d8e426a5c2f7936c307e07beea2dcf54300fae71986c43802a WatchSource:0}: Error finding container ce84963b65abc1d8e426a5c2f7936c307e07beea2dcf54300fae71986c43802a: Status 404 returned error can't find the container with id ce84963b65abc1d8e426a5c2f7936c307e07beea2dcf54300fae71986c43802a
Jul 14 19:38:26.135093 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.135065    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6571ea_7b0d_4671_a89b_e427c40cecc4.slice/crio-5aab78b5e3c79824de174e85fd1b6f00313f113a1a7c2b09d82a4dc3b307571a WatchSource:0}: Error finding container 5aab78b5e3c79824de174e85fd1b6f00313f113a1a7c2b09d82a4dc3b307571a: Status 404 returned error can't find the container with id 5aab78b5e3c79824de174e85fd1b6f00313f113a1a7c2b09d82a4dc3b307571a
Jul 14 19:38:26.136394 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.136069    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a83de8b_07d6_4adc_8f13_d8984c449f1a.slice/crio-a35a57d3a8f8aa77ea391935e7f413ca304b6539922da2e20e120eeb8bddd60e WatchSource:0}: Error finding container a35a57d3a8f8aa77ea391935e7f413ca304b6539922da2e20e120eeb8bddd60e: Status 404 returned error can't find the container with id a35a57d3a8f8aa77ea391935e7f413ca304b6539922da2e20e120eeb8bddd60e
Jul 14 19:38:26.137098 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:26.136993    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc028f4_1c3d_44d7_bf9a_8bd184c4f4ec.slice/crio-8736eb8dd27e743cfb638f3f9fc23baf670b7444691f33939f6ec45a0fe83461 WatchSource:0}: Error finding container 8736eb8dd27e743cfb638f3f9fc23baf670b7444691f33939f6ec45a0fe83461: Status 404 returned error can't find the container with id 8736eb8dd27e743cfb638f3f9fc23baf670b7444691f33939f6ec45a0fe83461
Jul 14 19:38:26.203921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.203617    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 19:33:25 +0000 UTC" deadline="2028-04-18 21:15:44.258084768 +0000 UTC"
Jul 14 19:38:26.203921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.203652    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15457h37m18.054437798s"
Jul 14 19:38:26.340188 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.340146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerStarted","Data":"a42db19b8733cd66bb36e7c78e2e152a402d0cfb67184bb73d55e5cda8384bde"}
Jul 14 19:38:26.341228 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.341200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n99s2" event={"ID":"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec","Type":"ContainerStarted","Data":"8736eb8dd27e743cfb638f3f9fc23baf670b7444691f33939f6ec45a0fe83461"}
Jul 14 19:38:26.342306 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.342273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqhxq" event={"ID":"7a83de8b-07d6-4adc-8f13-d8984c449f1a","Type":"ContainerStarted","Data":"a35a57d3a8f8aa77ea391935e7f413ca304b6539922da2e20e120eeb8bddd60e"}
Jul 14 19:38:26.343378 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.343356    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z826t" event={"ID":"cc6571ea-7b0d-4671-a89b-e427c40cecc4","Type":"ContainerStarted","Data":"5aab78b5e3c79824de174e85fd1b6f00313f113a1a7c2b09d82a4dc3b307571a"}
Jul 14 19:38:26.344322 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.344304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5p8x5" event={"ID":"13c49190-4865-4f90-a7f6-3400303c9879","Type":"ContainerStarted","Data":"e66df771c0dc3e6d3a9221453d181010291847215c2782d725824b90ab2996c8"}
Jul 14 19:38:26.345348 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.345328    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pbdmt" event={"ID":"c8e5430b-eadb-496c-b562-d9c3d6d58e6d","Type":"ContainerStarted","Data":"2fc755fe87c068a0ebd90a159f3dcd1eda1d14dcdad859d61f43b689edaf8538"}
Jul 14 19:38:26.347041 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.347018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal" event={"ID":"b7a9b0255c9ea7786828afe16280e2bd","Type":"ContainerStarted","Data":"f559b26b98ac8bd6f074c6ee1f4509fcab1cbc000f7cde3502e35a0d3a032de5"}
Jul 14 19:38:26.348141 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.348122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc" event={"ID":"a578d7c0-b23f-4285-984c-d1a2c313d167","Type":"ContainerStarted","Data":"ce84963b65abc1d8e426a5c2f7936c307e07beea2dcf54300fae71986c43802a"}
Jul 14 19:38:26.349126 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.349106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jdvp" event={"ID":"4250805b-0e7c-4d7d-8095-09ea841b2fa7","Type":"ContainerStarted","Data":"275e209fda1fec315a17545b1a19ab36b94dce5b4f02c7913d921bf9cbeb101e"}
Jul 14 19:38:26.350094 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.350072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"9d1e57a6a74fae47b6d7df17c115c0ff7a79a1c304e07b782234933903268994"}
Jul 14 19:38:26.774714 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.774023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:26.774714 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.774212    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 19:38:26.774714 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.774234    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 19:38:26.774714 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.774248    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zxxtv for pod openshift-network-diagnostics/network-check-target-49nh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:26.774714 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.774309    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv podName:8a29848a-06f0-42fb-96a6-4726020ea757 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:28.774291233 +0000 UTC m=+7.085990668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxxtv" (UniqueName: "kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv") pod "network-check-target-49nh2" (UID: "8a29848a-06f0-42fb-96a6-4726020ea757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:26.874950 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.874914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:26.875132 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:26.874978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:26.875193 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.875129    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:26.875242 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.875196    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:28.875174257 +0000 UTC m=+7.186873696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:26.890204 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.890119    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:26.890379 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:26.890264    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret podName:c7a2b6a9-bd74-458a-88a3-b7e5570767d4 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:28.890244455 +0000 UTC m=+7.201943889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret") pod "global-pull-secret-syncer-zdsp5" (UID: "c7a2b6a9-bd74-458a-88a3-b7e5570767d4") : object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:27.165460 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:27.165360    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 19:38:27.203898 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:27.203791    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 19:33:25 +0000 UTC" deadline="2028-04-07 14:09:01.83945427 +0000 UTC"
Jul 14 19:38:27.203898 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:27.203853    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15186h30m34.635605713s"
Jul 14 19:38:27.334245 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:27.333526    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:27.334245 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:27.333672    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:27.334245 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:27.334082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:27.334245 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:27.334196    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:27.334854 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:27.334701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:27.334854 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:27.334800    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:28.790249 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:28.789681    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:28.790249 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.789941    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 19:38:28.790249 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.789963    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 19:38:28.790249 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.789976    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zxxtv for pod openshift-network-diagnostics/network-check-target-49nh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:28.790249 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.790035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv podName:8a29848a-06f0-42fb-96a6-4726020ea757 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:32.790015723 +0000 UTC m=+11.101715175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxxtv" (UniqueName: "kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv") pod "network-check-target-49nh2" (UID: "8a29848a-06f0-42fb-96a6-4726020ea757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:28.891429 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:28.891011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:28.891429 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:28.891085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:28.891429 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.891280    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:28.891429 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.891335    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:28.891429 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.891355    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:32.891334907 +0000 UTC m=+11.203034347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:28.891429 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:28.891416    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret podName:c7a2b6a9-bd74-458a-88a3-b7e5570767d4 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:32.891382597 +0000 UTC m=+11.203082048 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret") pod "global-pull-secret-syncer-zdsp5" (UID: "c7a2b6a9-bd74-458a-88a3-b7e5570767d4") : object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:29.332524 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:29.332491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:29.332697 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:29.332639    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:29.333187 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:29.333036    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:29.333187 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:29.333146    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:29.333545 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:29.333426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:29.333545 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:29.333513    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:31.332098 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:31.332063    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:31.332603 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:31.332206    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:31.332603 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:31.332264    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:31.332603 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:31.332302    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:31.332603 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:31.332393    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:31.332603 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:31.332502    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:32.825770 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:32.825725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:32.826264 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.825923    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 19:38:32.826264 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.825950    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 19:38:32.826264 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.825965    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zxxtv for pod openshift-network-diagnostics/network-check-target-49nh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:32.826264 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.826035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv podName:8a29848a-06f0-42fb-96a6-4726020ea757 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:40.826015166 +0000 UTC m=+19.137714601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxxtv" (UniqueName: "kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv") pod "network-check-target-49nh2" (UID: "8a29848a-06f0-42fb-96a6-4726020ea757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:32.926618 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:32.926548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:32.926923 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:32.926691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:32.926923 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.926720    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:32.926923 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.926798    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret podName:c7a2b6a9-bd74-458a-88a3-b7e5570767d4 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:40.926777899 +0000 UTC m=+19.238477356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret") pod "global-pull-secret-syncer-zdsp5" (UID: "c7a2b6a9-bd74-458a-88a3-b7e5570767d4") : object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:32.926923 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.926816    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:32.926923 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:32.926871    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:40.926853237 +0000 UTC m=+19.238552685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:33.332958 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:33.332674    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:33.333214 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:33.333066    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:33.333516 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:33.333493    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:33.333632 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:33.333603    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:33.333697 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:33.333678    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:33.333775 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:33.333756    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:35.334056 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:35.334017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:35.334543 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:35.334189    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:35.334851 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:35.334685    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:35.338015 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:35.336914    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:35.338015 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:35.337020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:35.338015 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:35.337141    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:37.332505 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:37.332456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:37.332946 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:37.332456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:37.332946 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:37.332596    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:37.332946 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:37.332470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:37.332946 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:37.332736    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:37.332946 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:37.332813    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:39.331995 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:39.331954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:39.332487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:39.332003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:39.332487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:39.331954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:39.332487 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:39.332110    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:39.332487 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:39.332207    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:39.332487 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:39.332273    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:40.887654 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:40.887611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:40.888257 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.887838    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 19:38:40.888257 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.887872    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 19:38:40.888257 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.887888    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zxxtv for pod openshift-network-diagnostics/network-check-target-49nh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:40.888257 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.887961    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv podName:8a29848a-06f0-42fb-96a6-4726020ea757 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:56.887942884 +0000 UTC m=+35.199642317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxxtv" (UniqueName: "kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv") pod "network-check-target-49nh2" (UID: "8a29848a-06f0-42fb-96a6-4726020ea757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:40.988612 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:40.988567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:40.988795 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:40.988648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:40.988795 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.988750    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:40.988905 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.988814    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:56.988794856 +0000 UTC m=+35.300494290 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:40.988905 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.988751    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:40.989023 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:40.988918    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret podName:c7a2b6a9-bd74-458a-88a3-b7e5570767d4 nodeName:}" failed. No retries permitted until 2026-07-14 19:38:56.988897718 +0000 UTC m=+35.300597172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret") pod "global-pull-secret-syncer-zdsp5" (UID: "c7a2b6a9-bd74-458a-88a3-b7e5570767d4") : object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:41.332241 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:41.332198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:41.332459 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:41.332198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:41.332459 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:41.332342    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:41.332459 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:41.332448    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:41.332628 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:41.332206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:41.332628 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:41.332588    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:43.332785 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:43.332752    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:43.333302 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:43.332867    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:43.333302 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:43.332875    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:43.333302 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:43.332912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:43.333302 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:43.332998    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:43.333302 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:43.333113    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:44.897865 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:38:44.897820    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833569dec28ad50c40c5372b18ab9624.slice/crio-b8d3789ae6440ec498ca1f1c6e4b5c5f6d81c3556a12a85721216efc1902edb0 WatchSource:0}: Error finding container b8d3789ae6440ec498ca1f1c6e4b5c5f6d81c3556a12a85721216efc1902edb0: Status 404 returned error can't find the container with id b8d3789ae6440ec498ca1f1c6e4b5c5f6d81c3556a12a85721216efc1902edb0
Jul 14 19:38:45.332250 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.332053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:45.332472 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.332063    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:45.332472 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.332110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:45.332472 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:45.332361    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:45.332472 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:45.332465    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:45.332686 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:45.332546    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:45.391220 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.391184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5p8x5" event={"ID":"13c49190-4865-4f90-a7f6-3400303c9879","Type":"ContainerStarted","Data":"4861e413fe29a109cda401380c7a85a3d4bae22a5f04fc4370d1d2f44f18c25a"}
Jul 14 19:38:45.392641 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.392612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pbdmt" event={"ID":"c8e5430b-eadb-496c-b562-d9c3d6d58e6d","Type":"ContainerStarted","Data":"92483a23e1c31bd4b5bf6c8d296b8ee95d5d6b96956829b8952239757a8663e6"}
Jul 14 19:38:45.393657 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.393631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal" event={"ID":"833569dec28ad50c40c5372b18ab9624","Type":"ContainerStarted","Data":"b8d3789ae6440ec498ca1f1c6e4b5c5f6d81c3556a12a85721216efc1902edb0"}
Jul 14 19:38:45.394937 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.394916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc" event={"ID":"a578d7c0-b23f-4285-984c-d1a2c313d167","Type":"ContainerStarted","Data":"c0403a32b05e0fd826d1e888b9f98e83fd8ba0ccbfbe91f25c09f3365b0062b9"}
Jul 14 19:38:45.396496 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.396439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jdvp" event={"ID":"4250805b-0e7c-4d7d-8095-09ea841b2fa7","Type":"ContainerStarted","Data":"d5d7b792f8f93220ddbc42e3b76ad84fe7750f11ff9e6bd5f0147b5e99e9f3a8"}
Jul 14 19:38:45.398136 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.398116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"a2127d1214b42a320f9d4b09ed686ba2f50da86e40545e24f3f02dadd4aa3ce2"}
Jul 14 19:38:45.398235 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.398173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"8e5208bd41fecfa10f06ed28eb8a50b2b78cc79a9207cb726d28a0cbfc8f4eaa"}
Jul 14 19:38:45.399593 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.399571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerStarted","Data":"89a6fbe1c2dc014903ceec2697992c7721b6fba034450634082b00df70bf549e"}
Jul 14 19:38:45.400988 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.400966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n99s2" event={"ID":"7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec","Type":"ContainerStarted","Data":"4e8da3693ce700d55515ce17b8016e169ffdd98a127fecb6d413038059e44b46"}
Jul 14 19:38:45.402222 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.402202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z826t" event={"ID":"cc6571ea-7b0d-4671-a89b-e427c40cecc4","Type":"ContainerStarted","Data":"b5f2167923a640bed2fa16babe3f9d701a3040f0eb90602e13115b8298bc75ab"}
Jul 14 19:38:45.408053 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.408004    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5p8x5" podStartSLOduration=3.661190574 podStartE2EDuration="22.407964668s" totalImagesPullingTime="18.746774094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:45.407935462 +0000 UTC m=+23.719635083" watchObservedRunningTime="2026-07-14 19:38:45.407964668 +0000 UTC m=+23.719664125"
Jul 14 19:38:45.408620 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.408593    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.ec2.internal" podStartSLOduration=21.408583165 podStartE2EDuration="21.408583165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:26.377298031 +0000 UTC m=+4.688997488" watchObservedRunningTime="2026-07-14 19:38:45.408583165 +0000 UTC m=+23.720282622"
Jul 14 19:38:45.427054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.426993    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9jdvp" podStartSLOduration=3.66429137 podStartE2EDuration="22.426976563s" totalImagesPullingTime="18.762685193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:45.426796448 +0000 UTC m=+23.738495904" watchObservedRunningTime="2026-07-14 19:38:45.426976563 +0000 UTC m=+23.738676018"
Jul 14 19:38:45.447910 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.447835    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z826t" podStartSLOduration=3.704619311 podStartE2EDuration="22.447811742s" totalImagesPullingTime="18.743192431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:45.447546624 +0000 UTC m=+23.759246082" watchObservedRunningTime="2026-07-14 19:38:45.447811742 +0000 UTC m=+23.759511200"
Jul 14 19:38:45.464005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.463948    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n99s2" podStartSLOduration=3.986429342 podStartE2EDuration="22.463930423s" totalImagesPullingTime="18.477501081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:45.463741985 +0000 UTC m=+23.775441442" watchObservedRunningTime="2026-07-14 19:38:45.463930423 +0000 UTC m=+23.775629879"
Jul 14 19:38:45.483986 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.483925    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pbdmt" podStartSLOduration=3.676930279 podStartE2EDuration="22.483907758s" totalImagesPullingTime="18.806977479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:45.483807062 +0000 UTC m=+23.795506519" watchObservedRunningTime="2026-07-14 19:38:45.483907758 +0000 UTC m=+23.795607213"
Jul 14 19:38:45.526586 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:45.526554    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:46.406568 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:46.406526    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqhxq" event={"ID":"7a83de8b-07d6-4adc-8f13-d8984c449f1a","Type":"ContainerStarted","Data":"7c0cbab487a74cfd324d10d008a14185507b756cf81a5e10fe0fcdc2d7057f9b"}
Jul 14 19:38:46.408165 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:46.408134    2581 generic.go:356] "Generic (PLEG): container finished" podID="0777416d-4547-40b1-82f8-78911a02ad4a" containerID="89a6fbe1c2dc014903ceec2697992c7721b6fba034450634082b00df70bf549e" exitCode=0
Jul 14 19:38:46.408303 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:46.408189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerDied","Data":"89a6fbe1c2dc014903ceec2697992c7721b6fba034450634082b00df70bf549e"}
Jul 14 19:38:46.500327 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:46.500248    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lqhxq" podStartSLOduration=4.776093848 podStartE2EDuration="23.500221113s" totalImagesPullingTime="18.724127265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:46.453790993 +0000 UTC m=+24.765490449" watchObservedRunningTime="2026-07-14 19:38:46.500221113 +0000 UTC m=+24.811920583"
Jul 14 19:38:46.912546 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:46.912366    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 19:38:47.230639 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.230522    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T19:38:46.912538925Z","UUID":"471cac77-74b6-4e1d-9864-4cd5240178dd","Handler":null,"Name":"","Endpoint":""}
Jul 14 19:38:47.233065 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.233035    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 19:38:47.233219 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.233079    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 19:38:47.332225 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.332191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:47.332225 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.332201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:47.332501 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:47.332338    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:47.332501 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.332383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:47.332501 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:47.332471    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:47.332596 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:47.332538    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:47.411292 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.411256    2581 generic.go:356] "Generic (PLEG): container finished" podID="833569dec28ad50c40c5372b18ab9624" containerID="46602ffa9afb197a4f654ed527fe075a2fea2c395b0179ac0d3f26de440e588e" exitCode=0
Jul 14 19:38:47.411824 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.411336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal" event={"ID":"833569dec28ad50c40c5372b18ab9624","Type":"ContainerDied","Data":"46602ffa9afb197a4f654ed527fe075a2fea2c395b0179ac0d3f26de440e588e"}
Jul 14 19:38:47.413290 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:47.413266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc" event={"ID":"a578d7c0-b23f-4285-984c-d1a2c313d167","Type":"ContainerStarted","Data":"b6716963ea57988d0156e89fa4d2d3e2d770480423b380b6a13699920ffd8b76"}
Jul 14 19:38:48.134712 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:48.134673    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:48.135295 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:48.135277    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:48.417769 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:48.417686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal" event={"ID":"833569dec28ad50c40c5372b18ab9624","Type":"ContainerStarted","Data":"194ff49fbf0af1696cfb162b5f273ec6cb5ef4b7c713a807126087e160691412"}
Jul 14 19:38:48.418621 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:48.418600    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n99s2"
Jul 14 19:38:48.439943 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:48.439885    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.ec2.internal" podStartSLOduration=24.439865659 podStartE2EDuration="24.439865659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:48.439728317 +0000 UTC m=+26.751427774" watchObservedRunningTime="2026-07-14 19:38:48.439865659 +0000 UTC m=+26.751565117"
Jul 14 19:38:49.332703 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.332385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:49.332703 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:49.332546    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:49.332703 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.332618    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:49.333003 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:49.332717    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:49.333003 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.332784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:49.333003 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:49.332864    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:49.422775 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.422730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc" event={"ID":"a578d7c0-b23f-4285-984c-d1a2c313d167","Type":"ContainerStarted","Data":"89f85bfb88eeee7d3cba67bf35b665f23297bb914cbab00ea1a8517497a68b94"}
Jul 14 19:38:49.426395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.426357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"34aa8dd4053de973f9b2df3381a4f7e3ae3c9932a03c54d381705ce7af89d353"}
Jul 14 19:38:49.426576 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.426471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"00c91ecee16e1576f3238ffa03306fc9534b999133ff3beb231e5b5337a06960"}
Jul 14 19:38:49.426576 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.426530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"8c0432bc0df939d4459a57d211fe399e8038de463be6e145d31486bdd30213ec"}
Jul 14 19:38:49.426659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.426582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"697bcc8f09d284a6d934666d9039f0d384104726165000f96b2b704a573124c6"}
Jul 14 19:38:49.450620 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:49.450555    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wdcvc" podStartSLOduration=4.076100582 podStartE2EDuration="26.450531873s" totalImagesPullingTime="22.374431291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:49.450068611 +0000 UTC m=+27.761768058" watchObservedRunningTime="2026-07-14 19:38:49.450531873 +0000 UTC m=+27.762231331"
Jul 14 19:38:51.332056 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:51.332018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:51.332554 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:51.332018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:51.332554 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:51.332146    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:51.332554 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:51.332256    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:51.332554 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:51.332018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:51.332554 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:51.332356    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:52.436792 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:52.436759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"1f32c319f554f920ff3dd915e7dee638e0d5fb3a75b1ec7f1c77782fef871ecd"}
Jul 14 19:38:53.332386 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:53.332345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:53.332584 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:53.332452    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:53.332584 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:53.332465    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:53.332584 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:53.332487    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:53.332584 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:53.332563    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:53.332810 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:53.332664    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:53.441058 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:53.441013    2581 generic.go:356] "Generic (PLEG): container finished" podID="0777416d-4547-40b1-82f8-78911a02ad4a" containerID="1cfe5f0cf3001ffddfa7a1964b05f125901a693389de611828546db588dd6375" exitCode=0
Jul 14 19:38:53.441058 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:53.441055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerDied","Data":"1cfe5f0cf3001ffddfa7a1964b05f125901a693389de611828546db588dd6375"}
Jul 14 19:38:54.448526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:54.448229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" event={"ID":"92ada39f-8715-4093-b7ff-5db5af59b6b0","Type":"ContainerStarted","Data":"b706d0ec5484fd99ca0a4aef7875228e80f5bc7bebbd59d1046db21ec43ad984"}
Jul 14 19:38:54.448971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:54.448741    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:54.451539 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:54.450975    2581 generic.go:356] "Generic (PLEG): container finished" podID="0777416d-4547-40b1-82f8-78911a02ad4a" containerID="7a7108cd190a5bccc58d9941d2bf413c7a33b94c21175e8ce8439140f87867f8" exitCode=0
Jul 14 19:38:54.451539 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:54.451027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerDied","Data":"7a7108cd190a5bccc58d9941d2bf413c7a33b94c21175e8ce8439140f87867f8"}
Jul 14 19:38:54.465873 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:54.465811    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:54.485146 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:54.485102    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf" podStartSLOduration=9.468215719 podStartE2EDuration="31.485086481s" totalImagesPullingTime="22.016870762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:38:54.481849057 +0000 UTC m=+32.793548513" watchObservedRunningTime="2026-07-14 19:38:54.485086481 +0000 UTC m=+32.796785937"
Jul 14 19:38:55.332117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.331908    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:55.332317 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.331908    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:55.332317 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:55.332211    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:55.332317 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.331912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:55.332513 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:55.332260    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:55.332513 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:55.332350    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:55.456045 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.456008    2581 generic.go:356] "Generic (PLEG): container finished" podID="0777416d-4547-40b1-82f8-78911a02ad4a" containerID="7b0e7381931401c04f64d951b3be0ca889be585448a2795c424255847c4c5568" exitCode=0
Jul 14 19:38:55.456750 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.456079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerDied","Data":"7b0e7381931401c04f64d951b3be0ca889be585448a2795c424255847c4c5568"}
Jul 14 19:38:55.456750 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.456508    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:55.456750 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.456578    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:55.472127 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:55.472098    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:38:56.617689 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:56.617656    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zdsp5"]
Jul 14 19:38:56.618184 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:56.617784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:56.618184 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:56.617902    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:56.621005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:56.620974    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-49nh2"]
Jul 14 19:38:56.621142 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:56.621110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:56.621250 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:56.621221    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:56.621619 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:56.621598    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4jvmg"]
Jul 14 19:38:56.621713 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:56.621701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:56.621841 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:56.621802    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:38:56.903777 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:56.903743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:56.903978 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:56.903929    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 19:38:56.903978 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:56.903955    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 19:38:56.903978 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:56.903967    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zxxtv for pod openshift-network-diagnostics/network-check-target-49nh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:56.904125 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:56.904039    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv podName:8a29848a-06f0-42fb-96a6-4726020ea757 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:28.90401872 +0000 UTC m=+67.215718177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zxxtv" (UniqueName: "kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv") pod "network-check-target-49nh2" (UID: "8a29848a-06f0-42fb-96a6-4726020ea757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 19:38:57.004565 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:57.004537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:57.004693 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:57.004677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:57.004748 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:57.004681    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:57.004748 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:57.004727    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:57.004833 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:57.004781    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret podName:c7a2b6a9-bd74-458a-88a3-b7e5570767d4 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:29.004760559 +0000 UTC m=+67.316460003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret") pod "global-pull-secret-syncer-zdsp5" (UID: "c7a2b6a9-bd74-458a-88a3-b7e5570767d4") : object "kube-system"/"original-pull-secret" not registered
Jul 14 19:38:57.004833 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:57.004801    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:39:29.004790812 +0000 UTC m=+67.316490253 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 19:38:58.332492 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:58.332455    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:38:58.333013 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:58.332460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:38:58.333013 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:58.332564    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:38:58.333013 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:38:58.332460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:38:58.333013 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:58.332639    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:38:58.333013 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:38:58.332745    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:39:00.331775 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:00.331735    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:39:00.331775 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:00.331765    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:39:00.332419 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:00.331875    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdsp5" podUID="c7a2b6a9-bd74-458a-88a3-b7e5570767d4"
Jul 14 19:39:00.332419 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:00.331932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:39:00.332419 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:00.332071    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:39:00.332419 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:00.332182    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-49nh2" podUID="8a29848a-06f0-42fb-96a6-4726020ea757"
Jul 14 19:39:01.021802 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.021770    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-104.ec2.internal" event="NodeReady"
Jul 14 19:39:01.022001 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.021949    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 19:39:01.083918 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.083885    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nn5jk"]
Jul 14 19:39:01.103245 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.103210    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tpdjw"]
Jul 14 19:39:01.104264 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.104236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.108741 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.108713    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 19:39:01.108961 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.108937    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 19:39:01.109041 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.108995    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-774zx\""
Jul 14 19:39:01.114581 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.114562    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nn5jk"]
Jul 14 19:39:01.114680 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.114606    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tpdjw"]
Jul 14 19:39:01.114736 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.114696    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.117940 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.117917    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 19:39:01.118047 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.117973    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cpr6w\""
Jul 14 19:39:01.118832 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.118813    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 19:39:01.124451 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.124431    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 19:39:01.238552 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.238518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldrsv\" (UniqueName: \"kubernetes.io/projected/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-kube-api-access-ldrsv\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.238747 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.238582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-config-volume\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.238747 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.238610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-tmp-dir\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.238747 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.238664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k99pg\" (UniqueName: \"kubernetes.io/projected/99fa2d76-352e-406b-84e4-968d0ffd15da-kube-api-access-k99pg\") pod \"ingress-canary-tpdjw\" (UID: \"99fa2d76-352e-406b-84e4-968d0ffd15da\") " pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.238747 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.238715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.238912 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.238758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99fa2d76-352e-406b-84e4-968d0ffd15da-cert\") pod \"ingress-canary-tpdjw\" (UID: \"99fa2d76-352e-406b-84e4-968d0ffd15da\") " pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.339170 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99fa2d76-352e-406b-84e4-968d0ffd15da-cert\") pod \"ingress-canary-tpdjw\" (UID: \"99fa2d76-352e-406b-84e4-968d0ffd15da\") " pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.339170 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldrsv\" (UniqueName: \"kubernetes.io/projected/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-kube-api-access-ldrsv\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.339722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-config-volume\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.339722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-tmp-dir\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.339722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k99pg\" (UniqueName: \"kubernetes.io/projected/99fa2d76-352e-406b-84e4-968d0ffd15da-kube-api-access-k99pg\") pod \"ingress-canary-tpdjw\" (UID: \"99fa2d76-352e-406b-84e4-968d0ffd15da\") " pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.339722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.339722 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:01.339377    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:39:01.339722 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:01.339465    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:01.839444369 +0000 UTC m=+40.151143820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:39:01.339722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339678    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-tmp-dir\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.339988 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.339771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-config-volume\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.344549 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.344516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99fa2d76-352e-406b-84e4-968d0ffd15da-cert\") pod \"ingress-canary-tpdjw\" (UID: \"99fa2d76-352e-406b-84e4-968d0ffd15da\") " pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.351716 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.351689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldrsv\" (UniqueName: \"kubernetes.io/projected/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-kube-api-access-ldrsv\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.351856 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.351819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k99pg\" (UniqueName: \"kubernetes.io/projected/99fa2d76-352e-406b-84e4-968d0ffd15da-kube-api-access-k99pg\") pod \"ingress-canary-tpdjw\" (UID: \"99fa2d76-352e-406b-84e4-968d0ffd15da\") " pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.425004 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.424967    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tpdjw"
Jul 14 19:39:01.842177 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:01.842140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:01.842356 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:01.842286    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:39:01.842356 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:01.842350    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:02.842335469 +0000 UTC m=+41.154034902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:39:02.207693 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.207459    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tpdjw"]
Jul 14 19:39:02.213985 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:39:02.213955    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fa2d76_352e_406b_84e4_968d0ffd15da.slice/crio-8813bf63dbe4aef86380f18909c8aefda8342f5c755d5f7e4e22d3931ea81292 WatchSource:0}: Error finding container 8813bf63dbe4aef86380f18909c8aefda8342f5c755d5f7e4e22d3931ea81292: Status 404 returned error can't find the container with id 8813bf63dbe4aef86380f18909c8aefda8342f5c755d5f7e4e22d3931ea81292
Jul 14 19:39:02.332423 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.332369    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:39:02.332708 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.332459    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:39:02.332708 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.332478    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:39:02.335539 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.335513    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 19:39:02.336515 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.336496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cb2f5\""
Jul 14 19:39:02.336653 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.336519    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9cvvf\""
Jul 14 19:39:02.336653 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.336557    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 19:39:02.336653 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.336582    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 19:39:02.336653 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.336497    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 19:39:02.475421 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.475375    2581 generic.go:356] "Generic (PLEG): container finished" podID="0777416d-4547-40b1-82f8-78911a02ad4a" containerID="f3a2b1110e2c55599b8ee4d950ae494eed72667d63fed996d4f9a4669f6a509d" exitCode=0
Jul 14 19:39:02.476148 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.475447    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerDied","Data":"f3a2b1110e2c55599b8ee4d950ae494eed72667d63fed996d4f9a4669f6a509d"}
Jul 14 19:39:02.476762 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.476737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tpdjw" event={"ID":"99fa2d76-352e-406b-84e4-968d0ffd15da","Type":"ContainerStarted","Data":"8813bf63dbe4aef86380f18909c8aefda8342f5c755d5f7e4e22d3931ea81292"}
Jul 14 19:39:02.849111 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:02.849069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:02.849307 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:02.849194    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:39:02.849307 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:02.849252    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:04.849238076 +0000 UTC m=+43.160937510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:39:03.483872 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:03.483828    2581 generic.go:356] "Generic (PLEG): container finished" podID="0777416d-4547-40b1-82f8-78911a02ad4a" containerID="3472c93dae6c7c090c2983dec3b5c1d369a4f7d0273b8e0a2609cff97e1941bf" exitCode=0
Jul 14 19:39:03.484385 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:03.483977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerDied","Data":"3472c93dae6c7c090c2983dec3b5c1d369a4f7d0273b8e0a2609cff97e1941bf"}
Jul 14 19:39:04.492152 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:04.492115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v2sss" event={"ID":"0777416d-4547-40b1-82f8-78911a02ad4a","Type":"ContainerStarted","Data":"e9f6dd89d827e3e8318d5dfc5cd7c4687ca05a490b21da3598c42574c940ce8f"}
Jul 14 19:39:04.516702 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:04.516642    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v2sss" podStartSLOduration=7.683075715 podStartE2EDuration="41.516624775s" totalImagesPullingTime="33.83354906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:39:04.51622408 +0000 UTC m=+42.827923536" watchObservedRunningTime="2026-07-14 19:39:04.516624775 +0000 UTC m=+42.828324230"
Jul 14 19:39:04.865210 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:04.865124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:04.865363 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:04.865269    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:39:04.865363 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:04.865340    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:08.865322246 +0000 UTC m=+47.177021680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:39:06.497864 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:06.497824    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tpdjw" event={"ID":"99fa2d76-352e-406b-84e4-968d0ffd15da","Type":"ContainerStarted","Data":"6df8d59f00d134038b3f1008ce775dc334ee1271afa4146afff4fb0902ea3a69"}
Jul 14 19:39:06.515279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:06.515227    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tpdjw" podStartSLOduration=2.1200707579999998 podStartE2EDuration="5.515185194s" totalImagesPullingTime="3.395114436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:39:06.514773395 +0000 UTC m=+44.826472851" watchObservedRunningTime="2026-07-14 19:39:06.515185194 +0000 UTC m=+44.826884659"
Jul 14 19:39:08.893324 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:08.893281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:08.893730 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:08.893452    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:39:08.893730 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:08.893518    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:16.893502554 +0000 UTC m=+55.205201988 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:39:16.945906 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:16.945865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:16.946465 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:16.946038    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:39:16.946465 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:16.946106    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:39:32.946088493 +0000 UTC m=+71.257787926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:39:27.471717 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:27.471683    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmrvf"
Jul 14 19:39:28.934129 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:28.934059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:39:28.936967 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:28.936950    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 19:39:28.947801 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:28.947778    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 19:39:28.958728 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:28.958697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxxtv\" (UniqueName: \"kubernetes.io/projected/8a29848a-06f0-42fb-96a6-4726020ea757-kube-api-access-zxxtv\") pod \"network-check-target-49nh2\" (UID: \"8a29848a-06f0-42fb-96a6-4726020ea757\") " pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:39:29.034781 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.034741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:39:29.034948 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.034816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:39:29.037749 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.037730    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 19:39:29.037840 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.037784    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 19:39:29.045879 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:29.045850    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 14 19:39:29.045977 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:29.045924    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:40:33.045906168 +0000 UTC m=+131.357605603 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : secret "metrics-daemon-secret" not found
Jul 14 19:39:29.047890 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.047869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c7a2b6a9-bd74-458a-88a3-b7e5570767d4-original-pull-secret\") pod \"global-pull-secret-syncer-zdsp5\" (UID: \"c7a2b6a9-bd74-458a-88a3-b7e5570767d4\") " pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:39:29.056564 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.056540    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cb2f5\""
Jul 14 19:39:29.058279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.058266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdsp5"
Jul 14 19:39:29.063890 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.063873    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:39:29.195019 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.194937    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-49nh2"]
Jul 14 19:39:29.197751 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:39:29.197725    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a29848a_06f0_42fb_96a6_4726020ea757.slice/crio-23bec9aca30538610ca5af64a83bcfcb4d2ea600530ab56827a61f93aa2570c5 WatchSource:0}: Error finding container 23bec9aca30538610ca5af64a83bcfcb4d2ea600530ab56827a61f93aa2570c5: Status 404 returned error can't find the container with id 23bec9aca30538610ca5af64a83bcfcb4d2ea600530ab56827a61f93aa2570c5
Jul 14 19:39:29.208263 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.208237    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zdsp5"]
Jul 14 19:39:29.211094 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:39:29.211069    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a2b6a9_bd74_458a_88a3_b7e5570767d4.slice/crio-12cf9b12b5e905069f774e836490ace258d81a5cdc88c2cdb87ac9c1ea4af8d2 WatchSource:0}: Error finding container 12cf9b12b5e905069f774e836490ace258d81a5cdc88c2cdb87ac9c1ea4af8d2: Status 404 returned error can't find the container with id 12cf9b12b5e905069f774e836490ace258d81a5cdc88c2cdb87ac9c1ea4af8d2
Jul 14 19:39:29.553183 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.553147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-49nh2" event={"ID":"8a29848a-06f0-42fb-96a6-4726020ea757","Type":"ContainerStarted","Data":"23bec9aca30538610ca5af64a83bcfcb4d2ea600530ab56827a61f93aa2570c5"}
Jul 14 19:39:29.554185 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:29.554153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zdsp5" event={"ID":"c7a2b6a9-bd74-458a-88a3-b7e5570767d4","Type":"ContainerStarted","Data":"12cf9b12b5e905069f774e836490ace258d81a5cdc88c2cdb87ac9c1ea4af8d2"}
Jul 14 19:39:32.563165 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:32.563123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-49nh2" event={"ID":"8a29848a-06f0-42fb-96a6-4726020ea757","Type":"ContainerStarted","Data":"866e8a620ca362287398d8d987c7151f0f0067b5b0eb85b2ccd522ae773a893d"}
Jul 14 19:39:32.563599 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:32.563269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:39:32.581828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:32.581769    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-49nh2" podStartSLOduration=66.624394355 podStartE2EDuration="1m9.581751275s" totalImagesPullingTime="2.95735692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:39:32.581747555 +0000 UTC m=+70.893447011" watchObservedRunningTime="2026-07-14 19:39:32.581751275 +0000 UTC m=+70.893450818"
Jul 14 19:39:32.962702 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:32.962657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:39:32.962927 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:32.962865    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:39:32.963015 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:39:32.963001    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:40:04.962976381 +0000 UTC m=+103.274675816 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:39:34.569411 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:34.569354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zdsp5" event={"ID":"c7a2b6a9-bd74-458a-88a3-b7e5570767d4","Type":"ContainerStarted","Data":"f54994084a7100c024f82b87ac91a266d647372db145a44a6834b74aab1e9da4"}
Jul 14 19:39:34.585301 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:39:34.585235    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zdsp5" podStartSLOduration=66.742085716 podStartE2EDuration="1m11.585214876s" totalImagesPullingTime="4.84312916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:39:34.584865866 +0000 UTC m=+72.896565323" watchObservedRunningTime="2026-07-14 19:39:34.585214876 +0000 UTC m=+72.896914334"
Jul 14 19:40:03.567697 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:40:03.567577    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-49nh2"
Jul 14 19:40:04.990933 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:40:04.990886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:40:04.991346 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:40:04.991054    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:40:04.991346 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:40:04.991136    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:41:08.991118325 +0000 UTC m=+167.302817759 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:40:33.090625 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:40:33.090572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:40:33.091130 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:40:33.090713    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 14 19:40:33.091130 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:40:33.090779    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs podName:24cb2861-5df6-4718-9b15-38f935fa2397 nodeName:}" failed. No retries permitted until 2026-07-14 19:42:35.090764858 +0000 UTC m=+253.402464292 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs") pod "network-metrics-daemon-4jvmg" (UID: "24cb2861-5df6-4718-9b15-38f935fa2397") : secret "metrics-daemon-secret" not found
Jul 14 19:41:04.116861 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:04.116809    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nn5jk" podUID="0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f"
Jul 14 19:41:04.785362 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:04.785331    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:41:05.342587 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:05.342544    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4jvmg" podUID="24cb2861-5df6-4718-9b15-38f935fa2397"
Jul 14 19:41:09.043301 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:09.043245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:41:09.043743 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:09.043375    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 19:41:09.043743 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:09.043460    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls podName:0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f nodeName:}" failed. No retries permitted until 2026-07-14 19:43:11.043444295 +0000 UTC m=+289.355143728 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls") pod "dns-default-nn5jk" (UID: "0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f") : secret "dns-default-metrics-tls" not found
Jul 14 19:41:20.332179 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:20.332076    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:41:37.229476 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.229378    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"]
Jul 14 19:41:37.231344 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.231321    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"]
Jul 14 19:41:37.231509 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.231454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"
Jul 14 19:41:37.233208 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.233102    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.234137 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.234119    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r2qbm\""
Jul 14 19:41:37.234239 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.234209    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"]
Jul 14 19:41:37.235830 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.235811    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-wvfcn"]
Jul 14 19:41:37.235958 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.235941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"
Jul 14 19:41:37.237431 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.237412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.243442 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.243226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-l6rtg\""
Jul 14 19:41:37.243442 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.243294    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 14 19:41:37.244099 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.244077    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 14 19:41:37.244211 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.244075    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"]
Jul 14 19:41:37.245584 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.245547    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 14 19:41:37.245584 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.245551    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 14 19:41:37.245744 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.245628    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vxz74\""
Jul 14 19:41:37.245799 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.245753    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 14 19:41:37.246082 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.246065    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.248472 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.248450    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 14 19:41:37.248587 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.248450    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 14 19:41:37.248587 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.248518    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 19:41:37.248846 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.248760    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 19:41:37.248922 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.248873    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 14 19:41:37.248977 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.248949    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dcs87\""
Jul 14 19:41:37.249638 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.249612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 14 19:41:37.249638 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.249634    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 14 19:41:37.249794 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.249647    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 14 19:41:37.253114 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.253090    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 14 19:41:37.253260 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.253140    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-spqkv\""
Jul 14 19:41:37.255009 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.254988    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 14 19:41:37.258739 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.258712    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"]
Jul 14 19:41:37.264566 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.264543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"]
Jul 14 19:41:37.265478 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.265460    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"]
Jul 14 19:41:37.272236 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.272211    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-wvfcn"]
Jul 14 19:41:37.274877 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.274852    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"]
Jul 14 19:41:37.325943 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.325904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-serving-cert\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.326123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.325969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7216f409-33e7-434d-abf9-e8d9e953d3e2-serving-cert\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.326123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7216f409-33e7-434d-abf9-e8d9e953d3e2-trusted-ca\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.326123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6kth\" (UniqueName: \"kubernetes.io/projected/43c66065-d7e3-4761-bd3f-248bbfb2a0ac-kube-api-access-w6kth\") pod \"volume-data-source-validator-78495597f6-n4lqq\" (UID: \"43c66065-d7e3-4761-bd3f-248bbfb2a0ac\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"
Jul 14 19:41:37.326123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nr4v\" (UniqueName: \"kubernetes.io/projected/804e4dda-4e46-48f7-870b-65ca6a1fa4db-kube-api-access-7nr4v\") pod \"network-check-source-869dcfcdcd-n7rpt\" (UID: \"804e4dda-4e46-48f7-870b-65ca6a1fa4db\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"
Jul 14 19:41:37.326297 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7216f409-33e7-434d-abf9-e8d9e953d3e2-config\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.326297 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/79b168f0-d438-4064-9ee3-97bad5dfebfd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.326297 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckg2b\" (UniqueName: \"kubernetes.io/projected/7216f409-33e7-434d-abf9-e8d9e953d3e2-kube-api-access-ckg2b\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.326297 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326272    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.326466 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-config\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.326466 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvv4d\" (UniqueName: \"kubernetes.io/projected/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-kube-api-access-fvv4d\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.326466 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.326355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g62c4\" (UniqueName: \"kubernetes.io/projected/79b168f0-d438-4064-9ee3-97bad5dfebfd-kube-api-access-g62c4\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.335998 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.335967    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"]
Jul 14 19:41:37.338035 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.338017    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-q68tv"]
Jul 14 19:41:37.338168 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.338152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:37.339838 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.339816    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57656676ff-x95kl"]
Jul 14 19:41:37.339988 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.339970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:37.341419 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.341383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.342031 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.342012    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 14 19:41:37.342137 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.342030    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-w7wgs\""
Jul 14 19:41:37.342137 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.342104    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 14 19:41:37.342137 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.342121    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 14 19:41:37.343913 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.343898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2jlpr\""
Jul 14 19:41:37.345263 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.345197    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 14 19:41:37.351485 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.351465    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 14 19:41:37.351675 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.351651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 14 19:41:37.351987 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.351971    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xdxqx\""
Jul 14 19:41:37.352788 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.352767    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 14 19:41:37.353769 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.353749    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 14 19:41:37.359430 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.359391    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"]
Jul 14 19:41:37.362909 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.362885    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-q68tv"]
Jul 14 19:41:37.364890 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.364807    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 14 19:41:37.369272 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.369240    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57656676ff-x95kl"]
Jul 14 19:41:37.426684 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-serving-cert\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.426684 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-ca-trust-extracted\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.426921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-bound-sa-token\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.426921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6kth\" (UniqueName: \"kubernetes.io/projected/43c66065-d7e3-4761-bd3f-248bbfb2a0ac-kube-api-access-w6kth\") pod \"volume-data-source-validator-78495597f6-n4lqq\" (UID: \"43c66065-d7e3-4761-bd3f-248bbfb2a0ac\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"
Jul 14 19:41:37.426921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7216f409-33e7-434d-abf9-e8d9e953d3e2-trusted-ca\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.426921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426823    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-certificates\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.426921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr6h8\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-kube-api-access-jr6h8\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.426921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.426900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-installation-pull-secrets\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/de28e053-ed08-45f7-acd3-986ae8acd2fe-nginx-conf\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-trusted-ca\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7216f409-33e7-434d-abf9-e8d9e953d3e2-serving-cert\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nr4v\" (UniqueName: \"kubernetes.io/projected/804e4dda-4e46-48f7-870b-65ca6a1fa4db-kube-api-access-7nr4v\") pod \"network-check-source-869dcfcdcd-n7rpt\" (UID: \"804e4dda-4e46-48f7-870b-65ca6a1fa4db\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7216f409-33e7-434d-abf9-e8d9e953d3e2-config\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.427278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/79b168f0-d438-4064-9ee3-97bad5dfebfd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-image-registry-private-configuration\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gpjb\" (UniqueName: \"kubernetes.io/projected/c6937cda-845a-4b6f-881e-5b366d190d6e-kube-api-access-6gpjb\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckg2b\" (UniqueName: \"kubernetes.io/projected/7216f409-33e7-434d-abf9-e8d9e953d3e2-kube-api-access-ckg2b\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-config\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvv4d\" (UniqueName: \"kubernetes.io/projected/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-kube-api-access-fvv4d\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.427695 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g62c4\" (UniqueName: \"kubernetes.io/projected/79b168f0-d438-4064-9ee3-97bad5dfebfd-kube-api-access-g62c4\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.428091 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.427732    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:37.428091 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.427797    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls podName:79b168f0-d438-4064-9ee3-97bad5dfebfd nodeName:}" failed. No retries permitted until 2026-07-14 19:41:37.927777547 +0000 UTC m=+196.239476999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-222tg" (UID: "79b168f0-d438-4064-9ee3-97bad5dfebfd") : secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:37.428091 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.427941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7216f409-33e7-434d-abf9-e8d9e953d3e2-trusted-ca\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.428260 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.428248    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-config\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.428500 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.428478    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/79b168f0-d438-4064-9ee3-97bad5dfebfd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.428656 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.428632    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7216f409-33e7-434d-abf9-e8d9e953d3e2-config\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.429446 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.429396    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-serving-cert\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.429796 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.429775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7216f409-33e7-434d-abf9-e8d9e953d3e2-serving-cert\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.444127 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.444097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6kth\" (UniqueName: \"kubernetes.io/projected/43c66065-d7e3-4761-bd3f-248bbfb2a0ac-kube-api-access-w6kth\") pod \"volume-data-source-validator-78495597f6-n4lqq\" (UID: \"43c66065-d7e3-4761-bd3f-248bbfb2a0ac\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"
Jul 14 19:41:37.444246 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.444196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g62c4\" (UniqueName: \"kubernetes.io/projected/79b168f0-d438-4064-9ee3-97bad5dfebfd-kube-api-access-g62c4\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.455342 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.455314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nr4v\" (UniqueName: \"kubernetes.io/projected/804e4dda-4e46-48f7-870b-65ca6a1fa4db-kube-api-access-7nr4v\") pod \"network-check-source-869dcfcdcd-n7rpt\" (UID: \"804e4dda-4e46-48f7-870b-65ca6a1fa4db\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"
Jul 14 19:41:37.456386 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.456353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckg2b\" (UniqueName: \"kubernetes.io/projected/7216f409-33e7-434d-abf9-e8d9e953d3e2-kube-api-access-ckg2b\") pod \"console-operator-7866497494-wvfcn\" (UID: \"7216f409-33e7-434d-abf9-e8d9e953d3e2\") " pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.457899 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.457879    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvv4d\" (UniqueName: \"kubernetes.io/projected/ad53b9d1-10bc-47f5-9544-f69c83ec3e02-kube-api-access-fvv4d\") pod \"service-ca-operator-7695fc6c8c-nqs9n\" (UID: \"ad53b9d1-10bc-47f5-9544-f69c83ec3e02\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.527937 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.527824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-trusted-ca\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.527937 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.527893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:37.527937 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.527924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.527950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-image-registry-private-configuration\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.527971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gpjb\" (UniqueName: \"kubernetes.io/projected/c6937cda-845a-4b6f-881e-5b366d190d6e-kube-api-access-6gpjb\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-ca-trust-extracted\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-bound-sa-token\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.528058    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-certificates\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.528092    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr6h8\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-kube-api-access-jr6h8\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-installation-pull-secrets\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.528147    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls podName:c6937cda-845a-4b6f-881e-5b366d190d6e nodeName:}" failed. No retries permitted until 2026-07-14 19:41:38.028127799 +0000 UTC m=+196.339827236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls") pod "cluster-samples-operator-65db858894-7jqz5" (UID: "c6937cda-845a-4b6f-881e-5b366d190d6e") : secret "samples-operator-tls" not found
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.528185    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert podName:de28e053-ed08-45f7-acd3-986ae8acd2fe nodeName:}" failed. No retries permitted until 2026-07-14 19:41:38.028165663 +0000 UTC m=+196.339865116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q68tv" (UID: "de28e053-ed08-45f7-acd3-986ae8acd2fe") : secret "networking-console-plugin-cert" not found
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/de28e053-ed08-45f7-acd3-986ae8acd2fe-nginx-conf\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:37.528269 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528893 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.528369    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 19:41:37.528893 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.528383    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57656676ff-x95kl: secret "image-registry-tls" not found
Jul 14 19:41:37.528893 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.528447    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls podName:52f9adc9-9ebe-4728-bf7d-4e20a11887bb nodeName:}" failed. No retries permitted until 2026-07-14 19:41:38.028433226 +0000 UTC m=+196.340132678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls") pod "image-registry-57656676ff-x95kl" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb") : secret "image-registry-tls" not found
Jul 14 19:41:37.528893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-ca-trust-extracted\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.528893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-certificates\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.529071 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-trusted-ca\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.529071 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.528988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/de28e053-ed08-45f7-acd3-986ae8acd2fe-nginx-conf\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:37.530677 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.530653    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-installation-pull-secrets\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.530789 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.530764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-image-registry-private-configuration\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.546385 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.546351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"
Jul 14 19:41:37.548888 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.548859    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gpjb\" (UniqueName: \"kubernetes.io/projected/c6937cda-845a-4b6f-881e-5b366d190d6e-kube-api-access-6gpjb\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:37.550203 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.550181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr6h8\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-kube-api-access-jr6h8\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.550203 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.550194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-bound-sa-token\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:37.552114 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.552100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"
Jul 14 19:41:37.558745 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.558722    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"
Jul 14 19:41:37.565644 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.565607    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:37.707142 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.707107    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt"]
Jul 14 19:41:37.710562 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:41:37.710530    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod804e4dda_4e46_48f7_870b_65ca6a1fa4db.slice/crio-203631631f81831b3508057c96dba6997306f3d0be677cb2d67b90c086d5fd6c WatchSource:0}: Error finding container 203631631f81831b3508057c96dba6997306f3d0be677cb2d67b90c086d5fd6c: Status 404 returned error can't find the container with id 203631631f81831b3508057c96dba6997306f3d0be677cb2d67b90c086d5fd6c
Jul 14 19:41:37.865286 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.865188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt" event={"ID":"804e4dda-4e46-48f7-870b-65ca6a1fa4db","Type":"ContainerStarted","Data":"ef9536ebfc17ee39797d2ddc47f9e3e8208324db2022bb8be7f67419a1b7f50f"}
Jul 14 19:41:37.865286 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.865241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt" event={"ID":"804e4dda-4e46-48f7-870b-65ca6a1fa4db","Type":"ContainerStarted","Data":"203631631f81831b3508057c96dba6997306f3d0be677cb2d67b90c086d5fd6c"}
Jul 14 19:41:37.892395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.892340    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n7rpt" podStartSLOduration=0.892323302 podStartE2EDuration="892.323302ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:37.887949372 +0000 UTC m=+196.199648827" watchObservedRunningTime="2026-07-14 19:41:37.892323302 +0000 UTC m=+196.204022814"
Jul 14 19:41:37.931663 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.931611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:37.932093 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.931850    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:37.932093 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:37.931915    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls podName:79b168f0-d438-4064-9ee3-97bad5dfebfd nodeName:}" failed. No retries permitted until 2026-07-14 19:41:38.931896836 +0000 UTC m=+197.243596285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-222tg" (UID: "79b168f0-d438-4064-9ee3-97bad5dfebfd") : secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:37.933233 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.933207    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq"]
Jul 14 19:41:37.934659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.934625    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n"]
Jul 14 19:41:37.936071 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:41:37.936046    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c66065_d7e3_4761_bd3f_248bbfb2a0ac.slice/crio-ffc075ab9c63df47c710209ad7d598d0b04fc95e53e245f78a8068aadf51620a WatchSource:0}: Error finding container ffc075ab9c63df47c710209ad7d598d0b04fc95e53e245f78a8068aadf51620a: Status 404 returned error can't find the container with id ffc075ab9c63df47c710209ad7d598d0b04fc95e53e245f78a8068aadf51620a
Jul 14 19:41:37.936778 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:41:37.936757    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad53b9d1_10bc_47f5_9544_f69c83ec3e02.slice/crio-c7f72c4d352db5e5014a2330bae1559288f564e00500a98b873a2f67c70e3a1d WatchSource:0}: Error finding container c7f72c4d352db5e5014a2330bae1559288f564e00500a98b873a2f67c70e3a1d: Status 404 returned error can't find the container with id c7f72c4d352db5e5014a2330bae1559288f564e00500a98b873a2f67c70e3a1d
Jul 14 19:41:37.941824 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:37.941800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-wvfcn"]
Jul 14 19:41:37.945965 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:41:37.945938    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7216f409_33e7_434d_abf9_e8d9e953d3e2.slice/crio-6b1614bf7c27a8e4c6586837d3a145380c1c779cffd9d01d435fb56a3067509c WatchSource:0}: Error finding container 6b1614bf7c27a8e4c6586837d3a145380c1c779cffd9d01d435fb56a3067509c: Status 404 returned error can't find the container with id 6b1614bf7c27a8e4c6586837d3a145380c1c779cffd9d01d435fb56a3067509c
Jul 14 19:41:38.032001 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:38.031960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:38.032185 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:38.032052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:38.032185 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.032114    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 19:41:38.032185 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.032173    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert podName:de28e053-ed08-45f7-acd3-986ae8acd2fe nodeName:}" failed. No retries permitted until 2026-07-14 19:41:39.032160035 +0000 UTC m=+197.343859472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q68tv" (UID: "de28e053-ed08-45f7-acd3-986ae8acd2fe") : secret "networking-console-plugin-cert" not found
Jul 14 19:41:38.032185 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:38.032119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:38.032349 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.032214    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 19:41:38.032349 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.032236    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57656676ff-x95kl: secret "image-registry-tls" not found
Jul 14 19:41:38.032349 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.032286    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls podName:52f9adc9-9ebe-4728-bf7d-4e20a11887bb nodeName:}" failed. No retries permitted until 2026-07-14 19:41:39.032269102 +0000 UTC m=+197.343968549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls") pod "image-registry-57656676ff-x95kl" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb") : secret "image-registry-tls" not found
Jul 14 19:41:38.032349 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.032327    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 19:41:38.032519 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.032369    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls podName:c6937cda-845a-4b6f-881e-5b366d190d6e nodeName:}" failed. No retries permitted until 2026-07-14 19:41:39.032359128 +0000 UTC m=+197.344058567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls") pod "cluster-samples-operator-65db858894-7jqz5" (UID: "c6937cda-845a-4b6f-881e-5b366d190d6e") : secret "samples-operator-tls" not found
Jul 14 19:41:38.871010 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:38.870960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq" event={"ID":"43c66065-d7e3-4761-bd3f-248bbfb2a0ac","Type":"ContainerStarted","Data":"ffc075ab9c63df47c710209ad7d598d0b04fc95e53e245f78a8068aadf51620a"}
Jul 14 19:41:38.874570 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:38.874528    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n" event={"ID":"ad53b9d1-10bc-47f5-9544-f69c83ec3e02","Type":"ContainerStarted","Data":"c7f72c4d352db5e5014a2330bae1559288f564e00500a98b873a2f67c70e3a1d"}
Jul 14 19:41:38.877429 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:38.877161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wvfcn" event={"ID":"7216f409-33e7-434d-abf9-e8d9e953d3e2","Type":"ContainerStarted","Data":"6b1614bf7c27a8e4c6586837d3a145380c1c779cffd9d01d435fb56a3067509c"}
Jul 14 19:41:38.942144 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:38.941684    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:38.942144 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.941812    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:38.942144 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:38.941896    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls podName:79b168f0-d438-4064-9ee3-97bad5dfebfd nodeName:}" failed. No retries permitted until 2026-07-14 19:41:40.941874591 +0000 UTC m=+199.253574048 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-222tg" (UID: "79b168f0-d438-4064-9ee3-97bad5dfebfd") : secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:39.042805 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:39.042761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:39.042982 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:39.042853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:39.042982 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:39.042893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:39.042982 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:39.042948    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 19:41:39.042982 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:39.042974    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57656676ff-x95kl: secret "image-registry-tls" not found
Jul 14 19:41:39.043173 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:39.043030    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 19:41:39.043173 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:39.043042    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls podName:52f9adc9-9ebe-4728-bf7d-4e20a11887bb nodeName:}" failed. No retries permitted until 2026-07-14 19:41:41.043021226 +0000 UTC m=+199.354720672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls") pod "image-registry-57656676ff-x95kl" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb") : secret "image-registry-tls" not found
Jul 14 19:41:39.043173 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:39.043050    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 19:41:39.043173 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:39.043083    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert podName:de28e053-ed08-45f7-acd3-986ae8acd2fe nodeName:}" failed. No retries permitted until 2026-07-14 19:41:41.043067784 +0000 UTC m=+199.354767218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q68tv" (UID: "de28e053-ed08-45f7-acd3-986ae8acd2fe") : secret "networking-console-plugin-cert" not found
Jul 14 19:41:39.043173 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:39.043105    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls podName:c6937cda-845a-4b6f-881e-5b366d190d6e nodeName:}" failed. No retries permitted until 2026-07-14 19:41:41.043090686 +0000 UTC m=+199.354790138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls") pod "cluster-samples-operator-65db858894-7jqz5" (UID: "c6937cda-845a-4b6f-881e-5b366d190d6e") : secret "samples-operator-tls" not found
Jul 14 19:41:39.882333 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:39.882292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq" event={"ID":"43c66065-d7e3-4761-bd3f-248bbfb2a0ac","Type":"ContainerStarted","Data":"3572dafd6b2c4ef8f5f3d0b2ff24472ccc55aa0ad31f3df06144e3cb293002e0"}
Jul 14 19:41:39.899570 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:39.899510    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n4lqq" podStartSLOduration=1.7645075540000001 podStartE2EDuration="2.899484336s" totalImagesPullingTime="1.134976782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:39.898966879 +0000 UTC m=+198.210666336" watchObservedRunningTime="2026-07-14 19:41:39.899484336 +0000 UTC m=+198.211183795"
Jul 14 19:41:40.960167 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:40.959681    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:40.960167 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:40.959818    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:40.960167 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:40.959890    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls podName:79b168f0-d438-4064-9ee3-97bad5dfebfd nodeName:}" failed. No retries permitted until 2026-07-14 19:41:44.9598736 +0000 UTC m=+203.271573034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-222tg" (UID: "79b168f0-d438-4064-9ee3-97bad5dfebfd") : secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:41.060364 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.060326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:41.060364 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.060371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:41.060608 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.060436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:41.060608 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:41.060522    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 19:41:41.060608 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:41.060539    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 19:41:41.060608 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:41.060550    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57656676ff-x95kl: secret "image-registry-tls" not found
Jul 14 19:41:41.060608 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:41.060559    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 19:41:41.060608 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:41.060604    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls podName:c6937cda-845a-4b6f-881e-5b366d190d6e nodeName:}" failed. No retries permitted until 2026-07-14 19:41:45.060586438 +0000 UTC m=+203.372285874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls") pod "cluster-samples-operator-65db858894-7jqz5" (UID: "c6937cda-845a-4b6f-881e-5b366d190d6e") : secret "samples-operator-tls" not found
Jul 14 19:41:41.060785 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:41.060622    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert podName:de28e053-ed08-45f7-acd3-986ae8acd2fe nodeName:}" failed. No retries permitted until 2026-07-14 19:41:45.060613623 +0000 UTC m=+203.372313057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q68tv" (UID: "de28e053-ed08-45f7-acd3-986ae8acd2fe") : secret "networking-console-plugin-cert" not found
Jul 14 19:41:41.060785 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:41.060636    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls podName:52f9adc9-9ebe-4728-bf7d-4e20a11887bb nodeName:}" failed. No retries permitted until 2026-07-14 19:41:45.060630141 +0000 UTC m=+203.372329574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls") pod "image-registry-57656676ff-x95kl" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb") : secret "image-registry-tls" not found
Jul 14 19:41:41.889936 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.889895    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n" event={"ID":"ad53b9d1-10bc-47f5-9544-f69c83ec3e02","Type":"ContainerStarted","Data":"b9e1b992fde193578c3671fa02d4a72abb1779da1cba2a6b7fe84cc585304a36"}
Jul 14 19:41:41.891580 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.891556    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/0.log"
Jul 14 19:41:41.891708 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.891604    2581 generic.go:356] "Generic (PLEG): container finished" podID="7216f409-33e7-434d-abf9-e8d9e953d3e2" containerID="372868d28ea6d398a0cbe57e8e4b389ccf0dc1a20c7718c909de73290be41e81" exitCode=255
Jul 14 19:41:41.891708 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.891654    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wvfcn" event={"ID":"7216f409-33e7-434d-abf9-e8d9e953d3e2","Type":"ContainerDied","Data":"372868d28ea6d398a0cbe57e8e4b389ccf0dc1a20c7718c909de73290be41e81"}
Jul 14 19:41:41.891904 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.891891    2581 scope.go:117] "RemoveContainer" containerID="372868d28ea6d398a0cbe57e8e4b389ccf0dc1a20c7718c909de73290be41e81"
Jul 14 19:41:41.916360 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:41.916297    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n" podStartSLOduration=2.015627913 podStartE2EDuration="4.916280743s" totalImagesPullingTime="2.90065283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:41.916176428 +0000 UTC m=+200.227875898" watchObservedRunningTime="2026-07-14 19:41:41.916280743 +0000 UTC m=+200.227980198"
Jul 14 19:41:42.657306 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.657273    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ftrdr"]
Jul 14 19:41:42.660639 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.660620    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.664099 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.664071    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 19:41:42.664099 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.664097    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 19:41:42.664307 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.664129    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 19:41:42.664307 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.664133    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 19:41:42.665126 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.665113    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rt2ws\""
Jul 14 19:41:42.673311 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.673283    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftrdr"]
Jul 14 19:41:42.773175 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.773137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.773335 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.773191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-data-volume\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.773335 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.773227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.773428 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.773340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn4c2\" (UniqueName: \"kubernetes.io/projected/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-api-access-bn4c2\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.773474 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.773459    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-crio-socket\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.874365 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.874331    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn4c2\" (UniqueName: \"kubernetes.io/projected/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-api-access-bn4c2\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.874603 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.874393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-crio-socket\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.874603 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.874473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.874603 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.874490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-data-volume\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.874603 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.874507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.874603 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.874574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-crio-socket\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.874852 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:42.874588    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 19:41:42.874852 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:42.874599    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:42.874852 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:42.874685    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:43.37466371 +0000 UTC m=+201.686363168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : configmap "kube-rbac-proxy" not found
Jul 14 19:41:42.874852 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:42.874704    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:43.374692918 +0000 UTC m=+201.686392352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:42.874852 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.874798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-data-volume\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.884379 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.884346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn4c2\" (UniqueName: \"kubernetes.io/projected/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-api-access-bn4c2\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:42.896314 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.896290    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:41:42.896701 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.896688    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/0.log"
Jul 14 19:41:42.896754 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.896720    2581 generic.go:356] "Generic (PLEG): container finished" podID="7216f409-33e7-434d-abf9-e8d9e953d3e2" containerID="b09b7639775aeb5cb98f603ce9df38dd113b3d5368ddaad79618b354eaebb0a2" exitCode=255
Jul 14 19:41:42.896829 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.896808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wvfcn" event={"ID":"7216f409-33e7-434d-abf9-e8d9e953d3e2","Type":"ContainerDied","Data":"b09b7639775aeb5cb98f603ce9df38dd113b3d5368ddaad79618b354eaebb0a2"}
Jul 14 19:41:42.896885 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.896873    2581 scope.go:117] "RemoveContainer" containerID="372868d28ea6d398a0cbe57e8e4b389ccf0dc1a20c7718c909de73290be41e81"
Jul 14 19:41:42.897089 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:42.897063    2581 scope.go:117] "RemoveContainer" containerID="b09b7639775aeb5cb98f603ce9df38dd113b3d5368ddaad79618b354eaebb0a2"
Jul 14 19:41:42.897280 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:42.897259    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wvfcn_openshift-console-operator(7216f409-33e7-434d-abf9-e8d9e953d3e2)\"" pod="openshift-console-operator/console-operator-7866497494-wvfcn" podUID="7216f409-33e7-434d-abf9-e8d9e953d3e2"
Jul 14 19:41:43.379886 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.379840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:43.379886 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.379885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:43.380114 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:43.379939    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 19:41:43.380114 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:43.379992    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:43.380114 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:43.380020    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:44.379999306 +0000 UTC m=+202.691698740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : configmap "kube-rbac-proxy" not found
Jul 14 19:41:43.380114 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:43.380042    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:44.380032674 +0000 UTC m=+202.691732108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:43.869927 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.869783    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"]
Jul 14 19:41:43.873133 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.873107    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"
Jul 14 19:41:43.876915 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.876892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tht6z\""
Jul 14 19:41:43.877691 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.877675    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 14 19:41:43.878259 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.878244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 14 19:41:43.890064 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.890025    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"]
Jul 14 19:41:43.901767 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.901742    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:41:43.902075 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.902063    2581 scope.go:117] "RemoveContainer" containerID="b09b7639775aeb5cb98f603ce9df38dd113b3d5368ddaad79618b354eaebb0a2"
Jul 14 19:41:43.902222 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:43.902203    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wvfcn_openshift-console-operator(7216f409-33e7-434d-abf9-e8d9e953d3e2)\"" pod="openshift-console-operator/console-operator-7866497494-wvfcn" podUID="7216f409-33e7-434d-abf9-e8d9e953d3e2"
Jul 14 19:41:43.984686 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:43.984643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqv4t\" (UniqueName: \"kubernetes.io/projected/2dfcd161-8f36-49f1-bcc0-549c01a2da6f-kube-api-access-rqv4t\") pod \"migrator-7f59b54f6c-qxvsw\" (UID: \"2dfcd161-8f36-49f1-bcc0-549c01a2da6f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"
Jul 14 19:41:44.086220 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.086171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqv4t\" (UniqueName: \"kubernetes.io/projected/2dfcd161-8f36-49f1-bcc0-549c01a2da6f-kube-api-access-rqv4t\") pod \"migrator-7f59b54f6c-qxvsw\" (UID: \"2dfcd161-8f36-49f1-bcc0-549c01a2da6f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"
Jul 14 19:41:44.096968 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.096932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqv4t\" (UniqueName: \"kubernetes.io/projected/2dfcd161-8f36-49f1-bcc0-549c01a2da6f-kube-api-access-rqv4t\") pod \"migrator-7f59b54f6c-qxvsw\" (UID: \"2dfcd161-8f36-49f1-bcc0-549c01a2da6f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"
Jul 14 19:41:44.183388 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.183283    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"
Jul 14 19:41:44.324685 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.324648    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw"]
Jul 14 19:41:44.327734 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:41:44.327697    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dfcd161_8f36_49f1_bcc0_549c01a2da6f.slice/crio-6ec785d09530dcb8b2c35b7b2d183aa1e3565c0f3e92748acc90891d9228edf1 WatchSource:0}: Error finding container 6ec785d09530dcb8b2c35b7b2d183aa1e3565c0f3e92748acc90891d9228edf1: Status 404 returned error can't find the container with id 6ec785d09530dcb8b2c35b7b2d183aa1e3565c0f3e92748acc90891d9228edf1
Jul 14 19:41:44.388817 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.388782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:44.388817 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.388821    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:44.389039 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:44.388931    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 19:41:44.389039 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:44.388942    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:44.389039 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:44.389021    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:46.388997732 +0000 UTC m=+204.700697168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : configmap "kube-rbac-proxy" not found
Jul 14 19:41:44.389039 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:44.389039    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:46.389030512 +0000 UTC m=+204.700729947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:44.709665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.709639    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z826t_cc6571ea-7b0d-4671-a89b-e427c40cecc4/dns-node-resolver/0.log"
Jul 14 19:41:44.905872 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.905822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw" event={"ID":"2dfcd161-8f36-49f1-bcc0-549c01a2da6f","Type":"ContainerStarted","Data":"6ec785d09530dcb8b2c35b7b2d183aa1e3565c0f3e92748acc90891d9228edf1"}
Jul 14 19:41:44.993962 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:44.993849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:44.994113 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:44.993979    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:44.994113 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:44.994050    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls podName:79b168f0-d438-4064-9ee3-97bad5dfebfd nodeName:}" failed. No retries permitted until 2026-07-14 19:41:52.994032141 +0000 UTC m=+211.305731586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-222tg" (UID: "79b168f0-d438-4064-9ee3-97bad5dfebfd") : secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:45.094926 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:45.094892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:45.095112 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:45.094954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:45.095112 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:45.094982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:45.095112 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:45.095039    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 19:41:45.095112 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:45.095059    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57656676ff-x95kl: secret "image-registry-tls" not found
Jul 14 19:41:45.095112 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:45.095071    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 19:41:45.095112 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:45.095081    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 19:41:45.095112 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:45.095112    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls podName:52f9adc9-9ebe-4728-bf7d-4e20a11887bb nodeName:}" failed. No retries permitted until 2026-07-14 19:41:53.095098174 +0000 UTC m=+211.406797608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls") pod "image-registry-57656676ff-x95kl" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb") : secret "image-registry-tls" not found
Jul 14 19:41:45.095346 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:45.095126    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert podName:de28e053-ed08-45f7-acd3-986ae8acd2fe nodeName:}" failed. No retries permitted until 2026-07-14 19:41:53.09511965 +0000 UTC m=+211.406819083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q68tv" (UID: "de28e053-ed08-45f7-acd3-986ae8acd2fe") : secret "networking-console-plugin-cert" not found
Jul 14 19:41:45.095346 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:45.095136    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls podName:c6937cda-845a-4b6f-881e-5b366d190d6e nodeName:}" failed. No retries permitted until 2026-07-14 19:41:53.095131395 +0000 UTC m=+211.406830828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls") pod "cluster-samples-operator-65db858894-7jqz5" (UID: "c6937cda-845a-4b6f-881e-5b366d190d6e") : secret "samples-operator-tls" not found
Jul 14 19:41:45.306224 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:45.306141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5p8x5_13c49190-4865-4f90-a7f6-3400303c9879/node-ca/0.log"
Jul 14 19:41:46.308863 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:46.308832    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tpdjw_99fa2d76-352e-406b-84e4-968d0ffd15da/serve-healthcheck-canary/0.log"
Jul 14 19:41:46.406723 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:46.406682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:46.406905 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:46.406840    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:46.406905 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:46.406866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:46.406905 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:46.406901    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:50.406885794 +0000 UTC m=+208.718585259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:46.407088 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:46.406954    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 19:41:46.407088 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:46.407012    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:50.406996316 +0000 UTC m=+208.718695752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : configmap "kube-rbac-proxy" not found
Jul 14 19:41:47.566130 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:47.566032    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:47.566130 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:47.566090    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:41:47.566550 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:47.566396    2581 scope.go:117] "RemoveContainer" containerID="b09b7639775aeb5cb98f603ce9df38dd113b3d5368ddaad79618b354eaebb0a2"
Jul 14 19:41:47.566583 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:47.566569    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wvfcn_openshift-console-operator(7216f409-33e7-434d-abf9-e8d9e953d3e2)\"" pod="openshift-console-operator/console-operator-7866497494-wvfcn" podUID="7216f409-33e7-434d-abf9-e8d9e953d3e2"
Jul 14 19:41:47.918210 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:47.918111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw" event={"ID":"2dfcd161-8f36-49f1-bcc0-549c01a2da6f","Type":"ContainerStarted","Data":"bdfe1e236d17708502820448a73e6f14c64b8435a1c429905fcebd00b34dcd61"}
Jul 14 19:41:47.918210 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:47.918164    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw" event={"ID":"2dfcd161-8f36-49f1-bcc0-549c01a2da6f","Type":"ContainerStarted","Data":"c029a4e3df7304fe859dfeed8b158be134c16027357ca70bd6bdb605ecd44d46"}
Jul 14 19:41:50.441538 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:50.441497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:50.441538 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:50.441540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:50.441967 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:50.441618    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 19:41:50.441967 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:50.441639    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:50.441967 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:50.441685    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:58.44166994 +0000 UTC m=+216.753369374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : secret "insights-runtime-extractor-tls" not found
Jul 14 19:41:50.441967 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:50.441698    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm podName:6a2a1dc7-e4fb-4c41-bb67-69321e0a089b nodeName:}" failed. No retries permitted until 2026-07-14 19:41:58.441691988 +0000 UTC m=+216.753391423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftrdr" (UID: "6a2a1dc7-e4fb-4c41-bb67-69321e0a089b") : configmap "kube-rbac-proxy" not found
Jul 14 19:41:53.063466 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.063420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:41:53.063945 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:53.063522    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:53.063945 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:53.063610    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls podName:79b168f0-d438-4064-9ee3-97bad5dfebfd nodeName:}" failed. No retries permitted until 2026-07-14 19:42:09.063583551 +0000 UTC m=+227.375282998 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-222tg" (UID: "79b168f0-d438-4064-9ee3-97bad5dfebfd") : secret "cluster-monitoring-operator-tls" not found
Jul 14 19:41:53.164599 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.164562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:53.164752 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.164607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:41:53.164752 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.164683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:53.164832 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:53.164780    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 19:41:53.164871 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:41:53.164847    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert podName:de28e053-ed08-45f7-acd3-986ae8acd2fe nodeName:}" failed. No retries permitted until 2026-07-14 19:42:09.164830773 +0000 UTC m=+227.476530206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q68tv" (UID: "de28e053-ed08-45f7-acd3-986ae8acd2fe") : secret "networking-console-plugin-cert" not found
Jul 14 19:41:53.167097 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.167057    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6937cda-845a-4b6f-881e-5b366d190d6e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7jqz5\" (UID: \"c6937cda-845a-4b6f-881e-5b366d190d6e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:53.167097 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.167085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"image-registry-57656676ff-x95kl\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") " pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:53.248516 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.248474    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"
Jul 14 19:41:53.261488 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.261413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:53.389598 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.389535    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qxvsw" podStartSLOduration=7.479266098 podStartE2EDuration="10.389510227s" totalImagesPullingTime="2.910244129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:47.939939041 +0000 UTC m=+206.251638497" watchObservedRunningTime="2026-07-14 19:41:53.389510227 +0000 UTC m=+211.701209686"
Jul 14 19:41:53.390491 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.390467    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5"]
Jul 14 19:41:53.414929 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.414889    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57656676ff-x95kl"]
Jul 14 19:41:53.418356 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:41:53.418332    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f9adc9_9ebe_4728_bf7d_4e20a11887bb.slice/crio-8aeddc55a9d0051b31a8a93b6b26552fe13165eeb62168bcff833b0bf78e32e3 WatchSource:0}: Error finding container 8aeddc55a9d0051b31a8a93b6b26552fe13165eeb62168bcff833b0bf78e32e3: Status 404 returned error can't find the container with id 8aeddc55a9d0051b31a8a93b6b26552fe13165eeb62168bcff833b0bf78e32e3
Jul 14 19:41:53.937370 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.937323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57656676ff-x95kl" event={"ID":"52f9adc9-9ebe-4728-bf7d-4e20a11887bb","Type":"ContainerStarted","Data":"3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324"}
Jul 14 19:41:53.937583 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.937454    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:41:53.937583 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.937481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57656676ff-x95kl" event={"ID":"52f9adc9-9ebe-4728-bf7d-4e20a11887bb","Type":"ContainerStarted","Data":"8aeddc55a9d0051b31a8a93b6b26552fe13165eeb62168bcff833b0bf78e32e3"}
Jul 14 19:41:53.938455 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.938430    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5" event={"ID":"c6937cda-845a-4b6f-881e-5b366d190d6e","Type":"ContainerStarted","Data":"937fb3d5ec3f4d6c50f1ec6ddfb37464dcf436e7974f25c29229fc3f1ec68321"}
Jul 14 19:41:53.978643 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:53.978584    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57656676ff-x95kl" podStartSLOduration=16.978567702 podStartE2EDuration="16.978567702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:53.978275892 +0000 UTC m=+212.289975362" watchObservedRunningTime="2026-07-14 19:41:53.978567702 +0000 UTC m=+212.290267158"
Jul 14 19:41:55.945779 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:55.945744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5" event={"ID":"c6937cda-845a-4b6f-881e-5b366d190d6e","Type":"ContainerStarted","Data":"b197067acbcf0e10d869cc3c4a8cd549442d74dceb116a24956afc824b9f3958"}
Jul 14 19:41:55.946178 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:55.945797    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5" event={"ID":"c6937cda-845a-4b6f-881e-5b366d190d6e","Type":"ContainerStarted","Data":"9de586dcd8c4c09000d3c01c331dca0e978a9784a4675e07b8da175ac961975f"}
Jul 14 19:41:55.970467 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:55.970385    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7jqz5" podStartSLOduration=17.012045219 podStartE2EDuration="18.970367063s" totalImagesPullingTime="1.958321844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:41:55.96986924 +0000 UTC m=+214.281568695" watchObservedRunningTime="2026-07-14 19:41:55.970367063 +0000 UTC m=+214.282066519"
Jul 14 19:41:58.512821 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:58.512775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:58.512821 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:58.512818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:58.513333 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:58.513303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:58.515164 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:58.515132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a1dc7-e4fb-4c41-bb67-69321e0a089b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftrdr\" (UID: \"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b\") " pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:58.570387 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:58.570346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftrdr"
Jul 14 19:41:58.702796 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:58.702659    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftrdr"]
Jul 14 19:41:58.705362 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:41:58.705339    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2a1dc7_e4fb_4c41_bb67_69321e0a089b.slice/crio-d1450715e7749d64594d2c27034fdb1a684514c64644a660168f4ab70929de43 WatchSource:0}: Error finding container d1450715e7749d64594d2c27034fdb1a684514c64644a660168f4ab70929de43: Status 404 returned error can't find the container with id d1450715e7749d64594d2c27034fdb1a684514c64644a660168f4ab70929de43
Jul 14 19:41:58.955363 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:41:58.955327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftrdr" event={"ID":"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b","Type":"ContainerStarted","Data":"d1450715e7749d64594d2c27034fdb1a684514c64644a660168f4ab70929de43"}
Jul 14 19:42:02.332530 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:02.332496    2581 scope.go:117] "RemoveContainer" containerID="b09b7639775aeb5cb98f603ce9df38dd113b3d5368ddaad79618b354eaebb0a2"
Jul 14 19:42:02.973938 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:02.973906    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:42:02.974134 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:02.973997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wvfcn" event={"ID":"7216f409-33e7-434d-abf9-e8d9e953d3e2","Type":"ContainerStarted","Data":"7aa619318550c057fd1798b4c3bbfd85f00c5fb892dc7a92e84b90116cbf9755"}
Jul 14 19:42:02.974383 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:02.974340    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:42:02.994757 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:02.994665    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-wvfcn" podStartSLOduration=23.101630228 podStartE2EDuration="25.99464262s" totalImagesPullingTime="2.893012392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:02.993436228 +0000 UTC m=+221.305135687" watchObservedRunningTime="2026-07-14 19:42:02.99464262 +0000 UTC m=+221.306342075"
Jul 14 19:42:03.713513 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:03.713479    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-wvfcn"
Jul 14 19:42:05.988666 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:05.988620    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftrdr" event={"ID":"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b","Type":"ContainerStarted","Data":"e09b175522639f0bf9a1b3483a938e6404cf921ef99628bf94a4d259f54eba9b"}
Jul 14 19:42:06.993116 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:06.993026    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftrdr" event={"ID":"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b","Type":"ContainerStarted","Data":"ba0b39f9688a866d96541f43e2fa2e641c99bfee4fbef944ce3230a961f61c03"}
Jul 14 19:42:07.582610 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.582575    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57656676ff-x95kl"]
Jul 14 19:42:07.590904 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.590504    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-dgcpn"]
Jul 14 19:42:07.594307 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.594280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-dgcpn"
Jul 14 19:42:07.599778 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.599485    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 14 19:42:07.599778 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.599538    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 14 19:42:07.599778 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.599566    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-k9tjd\""
Jul 14 19:42:07.610555 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.610482    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-dgcpn"]
Jul 14 19:42:07.694063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.694022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qlsd\" (UniqueName: \"kubernetes.io/projected/24d3d98f-c435-40da-8683-3d266890b3ff-kube-api-access-5qlsd\") pod \"downloads-68c9cbb497-dgcpn\" (UID: \"24d3d98f-c435-40da-8683-3d266890b3ff\") " pod="openshift-console/downloads-68c9cbb497-dgcpn"
Jul 14 19:42:07.712803 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.712763    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-676f444bdd-6wthl"]
Jul 14 19:42:07.716530 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.716502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.795754 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.795716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-bound-sa-token\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.795950 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.795771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dd6c5539-7faf-4701-b16b-237a56f2ef5e-installation-pull-secrets\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.795950 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.795890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjqcx\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-kube-api-access-qjqcx\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.795950 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.795934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-registry-tls\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.796114 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.795964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dd6c5539-7faf-4701-b16b-237a56f2ef5e-registry-certificates\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.796114 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.795991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dd6c5539-7faf-4701-b16b-237a56f2ef5e-image-registry-private-configuration\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.796114 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.796024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qlsd\" (UniqueName: \"kubernetes.io/projected/24d3d98f-c435-40da-8683-3d266890b3ff-kube-api-access-5qlsd\") pod \"downloads-68c9cbb497-dgcpn\" (UID: \"24d3d98f-c435-40da-8683-3d266890b3ff\") " pod="openshift-console/downloads-68c9cbb497-dgcpn"
Jul 14 19:42:07.796114 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.796053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dd6c5539-7faf-4701-b16b-237a56f2ef5e-ca-trust-extracted\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.796294 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.796142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd6c5539-7faf-4701-b16b-237a56f2ef5e-trusted-ca\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.810053 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.810012    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676f444bdd-6wthl"]
Jul 14 19:42:07.814147 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.814079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qlsd\" (UniqueName: \"kubernetes.io/projected/24d3d98f-c435-40da-8683-3d266890b3ff-kube-api-access-5qlsd\") pod \"downloads-68c9cbb497-dgcpn\" (UID: \"24d3d98f-c435-40da-8683-3d266890b3ff\") " pod="openshift-console/downloads-68c9cbb497-dgcpn"
Jul 14 19:42:07.897122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dd6c5539-7faf-4701-b16b-237a56f2ef5e-registry-certificates\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dd6c5539-7faf-4701-b16b-237a56f2ef5e-image-registry-private-configuration\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897122 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dd6c5539-7faf-4701-b16b-237a56f2ef5e-ca-trust-extracted\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897418 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd6c5539-7faf-4701-b16b-237a56f2ef5e-trusted-ca\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897418 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-bound-sa-token\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897418 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dd6c5539-7faf-4701-b16b-237a56f2ef5e-installation-pull-secrets\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897597 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjqcx\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-kube-api-access-qjqcx\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897597 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dd6c5539-7faf-4701-b16b-237a56f2ef5e-ca-trust-extracted\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.897597 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.897529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-registry-tls\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.898064 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.898036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dd6c5539-7faf-4701-b16b-237a56f2ef5e-registry-certificates\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.898322 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.898299    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd6c5539-7faf-4701-b16b-237a56f2ef5e-trusted-ca\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.900590 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.900568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dd6c5539-7faf-4701-b16b-237a56f2ef5e-image-registry-private-configuration\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.900590 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.900583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-registry-tls\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.900746 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.900566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dd6c5539-7faf-4701-b16b-237a56f2ef5e-installation-pull-secrets\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.906665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.906642    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-dgcpn"
Jul 14 19:42:07.906866 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.906837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-bound-sa-token\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:07.912112 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:07.912084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjqcx\" (UniqueName: \"kubernetes.io/projected/dd6c5539-7faf-4701-b16b-237a56f2ef5e-kube-api-access-qjqcx\") pod \"image-registry-676f444bdd-6wthl\" (UID: \"dd6c5539-7faf-4701-b16b-237a56f2ef5e\") " pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:08.028157 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:08.027799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:08.077583 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:08.077549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-dgcpn"]
Jul 14 19:42:08.081460 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:08.081422    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d3d98f_c435_40da_8683_3d266890b3ff.slice/crio-ff48ebd691127109e63d2166a2a16ea6795adf74a3847f1e49d918801a774605 WatchSource:0}: Error finding container ff48ebd691127109e63d2166a2a16ea6795adf74a3847f1e49d918801a774605: Status 404 returned error can't find the container with id ff48ebd691127109e63d2166a2a16ea6795adf74a3847f1e49d918801a774605
Jul 14 19:42:08.204648 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:08.204610    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676f444bdd-6wthl"]
Jul 14 19:42:08.208134 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:08.208100    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6c5539_7faf_4701_b16b_237a56f2ef5e.slice/crio-4e5fa1e52a5c7119475ab0d20d7907636c0769cc78c8d4f37af903e1b86e09c5 WatchSource:0}: Error finding container 4e5fa1e52a5c7119475ab0d20d7907636c0769cc78c8d4f37af903e1b86e09c5: Status 404 returned error can't find the container with id 4e5fa1e52a5c7119475ab0d20d7907636c0769cc78c8d4f37af903e1b86e09c5
Jul 14 19:42:09.002392 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.002346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" event={"ID":"dd6c5539-7faf-4701-b16b-237a56f2ef5e","Type":"ContainerStarted","Data":"97c19326423aa00be58f658c913718c30a4f1d4d7e98b354338478e83f7f83cb"}
Jul 14 19:42:09.002610 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.002436    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" event={"ID":"dd6c5539-7faf-4701-b16b-237a56f2ef5e","Type":"ContainerStarted","Data":"4e5fa1e52a5c7119475ab0d20d7907636c0769cc78c8d4f37af903e1b86e09c5"}
Jul 14 19:42:09.002610 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.002467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:09.008556 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.008519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-dgcpn" event={"ID":"24d3d98f-c435-40da-8683-3d266890b3ff","Type":"ContainerStarted","Data":"ff48ebd691127109e63d2166a2a16ea6795adf74a3847f1e49d918801a774605"}
Jul 14 19:42:09.108958 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.108900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:42:09.111908 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.111882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/79b168f0-d438-4064-9ee3-97bad5dfebfd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-222tg\" (UID: \"79b168f0-d438-4064-9ee3-97bad5dfebfd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:42:09.210277 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.210230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:42:09.213016 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.212988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/de28e053-ed08-45f7-acd3-986ae8acd2fe-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q68tv\" (UID: \"de28e053-ed08-45f7-acd3-986ae8acd2fe\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:42:09.371912 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.371833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"
Jul 14 19:42:09.455457 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.455394    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv"
Jul 14 19:42:09.536906 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.536817    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podStartSLOduration=2.536795943 podStartE2EDuration="2.536795943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:09.024852713 +0000 UTC m=+227.336552170" watchObservedRunningTime="2026-07-14 19:42:09.536795943 +0000 UTC m=+227.848495400"
Jul 14 19:42:09.538828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.538803    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg"]
Jul 14 19:42:09.540961 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:09.540936    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79b168f0_d438_4064_9ee3_97bad5dfebfd.slice/crio-b83dad1c4ecbf70c0d3b2ad66ad7825d82dc0265b64ef1e611585d8c36747693 WatchSource:0}: Error finding container b83dad1c4ecbf70c0d3b2ad66ad7825d82dc0265b64ef1e611585d8c36747693: Status 404 returned error can't find the container with id b83dad1c4ecbf70c0d3b2ad66ad7825d82dc0265b64ef1e611585d8c36747693
Jul 14 19:42:09.601634 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:09.601602    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-q68tv"]
Jul 14 19:42:09.604749 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:09.604715    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde28e053_ed08_45f7_acd3_986ae8acd2fe.slice/crio-d40c3aaedb5e8776a1a068d4265e43f4baa5854a488474b008c9485f3998f44d WatchSource:0}: Error finding container d40c3aaedb5e8776a1a068d4265e43f4baa5854a488474b008c9485f3998f44d: Status 404 returned error can't find the container with id d40c3aaedb5e8776a1a068d4265e43f4baa5854a488474b008c9485f3998f44d
Jul 14 19:42:10.015640 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:10.015595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftrdr" event={"ID":"6a2a1dc7-e4fb-4c41-bb67-69321e0a089b","Type":"ContainerStarted","Data":"3aca374eb744a9236e6ac5962df09aef00e5bec15a9fa35b0ad8351643df5ea6"}
Jul 14 19:42:10.016945 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:10.016912    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv" event={"ID":"de28e053-ed08-45f7-acd3-986ae8acd2fe","Type":"ContainerStarted","Data":"d40c3aaedb5e8776a1a068d4265e43f4baa5854a488474b008c9485f3998f44d"}
Jul 14 19:42:10.018451 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:10.018420    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg" event={"ID":"79b168f0-d438-4064-9ee3-97bad5dfebfd","Type":"ContainerStarted","Data":"b83dad1c4ecbf70c0d3b2ad66ad7825d82dc0265b64ef1e611585d8c36747693"}
Jul 14 19:42:10.037816 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:10.037761    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ftrdr" podStartSLOduration=17.415121801 podStartE2EDuration="28.037736687s" totalImagesPullingTime="10.622614886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 19:41:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:10.03666241 +0000 UTC m=+228.348361867" watchObservedRunningTime="2026-07-14 19:42:10.037736687 +0000 UTC m=+228.349436149"
Jul 14 19:42:13.033346 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:13.033292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv" event={"ID":"de28e053-ed08-45f7-acd3-986ae8acd2fe","Type":"ContainerStarted","Data":"5c6083e2c926a614b3762d9b941882737e451d50308a4b3bd55ea777a8ccd772"}
Jul 14 19:42:13.035117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:13.035087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg" event={"ID":"79b168f0-d438-4064-9ee3-97bad5dfebfd","Type":"ContainerStarted","Data":"f25b3a967858787a0c3ded91ab1844d5ff53624f4c3c73a9993ccc4bcd7c4b69"}
Jul 14 19:42:13.052377 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:13.052322    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-q68tv" podStartSLOduration=33.58430121 podStartE2EDuration="36.052305626s" totalImagesPullingTime="2.468004416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:13.051057423 +0000 UTC m=+231.362756882" watchObservedRunningTime="2026-07-14 19:42:13.052305626 +0000 UTC m=+231.364005084"
Jul 14 19:42:13.067037 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:13.066975    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-222tg" podStartSLOduration=33.533851027 podStartE2EDuration="36.066952517s" totalImagesPullingTime="2.53310149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:13.066101484 +0000 UTC m=+231.377800942" watchObservedRunningTime="2026-07-14 19:42:13.066952517 +0000 UTC m=+231.378651974"
Jul 14 19:42:17.588911 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:17.588863    2581 patch_prober.go:28] interesting pod/image-registry-57656676ff-x95kl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:17.589342 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:17.588930    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-57656676ff-x95kl" podUID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:20.158960 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.158926    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"]
Jul 14 19:42:20.161827 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.161804    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.164980 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.164954    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 14 19:42:20.165121 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.164975    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 14 19:42:20.166417 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.165720    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 14 19:42:20.167326 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.166796    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-c7kg8\""
Jul 14 19:42:20.171076 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.171048    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 19:42:20.179942 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.179907    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"]
Jul 14 19:42:20.185378 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.185351    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qvn6s"]
Jul 14 19:42:20.188004 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.187982    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.190744 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.190719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 19:42:20.192140 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.192114    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mtsrq\""
Jul 14 19:42:20.192702 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.192681    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 19:42:20.192805 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.192721    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 19:42:20.311522 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.311734 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0870fac-f247-4234-be87-42613c05d2a2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.311734 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-tls\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.311734 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99cd6dd-2793-41b4-87b3-bd56e4450401-metrics-client-ca\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.311734 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.311931 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-textfile\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.311931 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311777    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-root\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.311931 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311846    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.311931 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-wtmp\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.312109 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.311950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0870fac-f247-4234-be87-42613c05d2a2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.312109 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.312032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-accelerators-collector-config\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.312109 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.312095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7skb\" (UniqueName: \"kubernetes.io/projected/e0870fac-f247-4234-be87-42613c05d2a2-kube-api-access-s7skb\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.312235 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.312121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lshb\" (UniqueName: \"kubernetes.io/projected/f99cd6dd-2793-41b4-87b3-bd56e4450401-kube-api-access-7lshb\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.312235 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.312160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.312235 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.312192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-sys\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.412766 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.412663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0870fac-f247-4234-be87-42613c05d2a2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.412766 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.412743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-tls\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413007 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.412772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99cd6dd-2793-41b4-87b3-bd56e4450401-metrics-client-ca\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413007 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.412805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.413007 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.412846    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-textfile\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413007 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.412873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-root\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413007 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:42:20.412914    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 14 19:42:20.413007 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.412942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-root\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413007 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:42:20.412990    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-tls podName:f99cd6dd-2793-41b4-87b3-bd56e4450401 nodeName:}" failed. No retries permitted until 2026-07-14 19:42:20.912966695 +0000 UTC m=+239.224666143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-tls") pod "node-exporter-qvn6s" (UID: "f99cd6dd-2793-41b4-87b3-bd56e4450401") : secret "node-exporter-tls" not found
Jul 14 19:42:20.413331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.413331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-wtmp\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0870fac-f247-4234-be87-42613c05d2a2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.413331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-accelerators-collector-config\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7skb\" (UniqueName: \"kubernetes.io/projected/e0870fac-f247-4234-be87-42613c05d2a2-kube-api-access-s7skb\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.413331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413262    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-textfile\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lshb\" (UniqueName: \"kubernetes.io/projected/f99cd6dd-2793-41b4-87b3-bd56e4450401-kube-api-access-7lshb\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.413692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-sys\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-wtmp\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.413692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99cd6dd-2793-41b4-87b3-bd56e4450401-metrics-client-ca\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.413692 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:42:20.413694    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 14 19:42:20.414042 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:42:20.413783    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-tls podName:e0870fac-f247-4234-be87-42613c05d2a2 nodeName:}" failed. No retries permitted until 2026-07-14 19:42:20.913767651 +0000 UTC m=+239.225467096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-vpkrs" (UID: "e0870fac-f247-4234-be87-42613c05d2a2") : secret "kube-state-metrics-tls" not found
Jul 14 19:42:20.414042 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-accelerators-collector-config\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.414042 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0870fac-f247-4234-be87-42613c05d2a2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.414042 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f99cd6dd-2793-41b4-87b3-bd56e4450401-sys\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.414042 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.413971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e0870fac-f247-4234-be87-42613c05d2a2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.416023 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.415993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.416213 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.416195    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.427727 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.427697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lshb\" (UniqueName: \"kubernetes.io/projected/f99cd6dd-2793-41b4-87b3-bd56e4450401-kube-api-access-7lshb\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.438011 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.437973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7skb\" (UniqueName: \"kubernetes.io/projected/e0870fac-f247-4234-be87-42613c05d2a2-kube-api-access-s7skb\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.918010 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.917963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:20.918220 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.918071    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-tls\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.920885 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.920856    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f99cd6dd-2793-41b4-87b3-bd56e4450401-node-exporter-tls\") pod \"node-exporter-qvn6s\" (UID: \"f99cd6dd-2793-41b4-87b3-bd56e4450401\") " pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:20.921033 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:20.920917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e0870fac-f247-4234-be87-42613c05d2a2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-vpkrs\" (UID: \"e0870fac-f247-4234-be87-42613c05d2a2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:21.076623 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:21.076586    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"
Jul 14 19:42:21.100376 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:21.100339    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qvn6s"
Jul 14 19:42:23.249772 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.249734    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"]
Jul 14 19:42:23.252576 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.252547    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.257785 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.257746    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 14 19:42:23.257972 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.257923    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 14 19:42:23.258174 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.258154    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 14 19:42:23.258270 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.258251    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 14 19:42:23.258331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.258313    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-29mrf\""
Jul 14 19:42:23.258387 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.258372    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 14 19:42:23.258694 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.258676    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fil5ed73ko84a\""
Jul 14 19:42:23.276259 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.276226    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"]
Jul 14 19:42:23.335934 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.335896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-grpc-tls\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.336112 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.335961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.336112 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.336068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fbb8f4e-55da-4da4-a6a6-34414e099976-metrics-client-ca\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.336252 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.336166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.336252 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.336197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tstll\" (UniqueName: \"kubernetes.io/projected/0fbb8f4e-55da-4da4-a6a6-34414e099976-kube-api-access-tstll\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.336252 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.336226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.336426 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.336348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.336498 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.336447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-tls\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.436985 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.436935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.437195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.437009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-tls\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.437195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.437033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-grpc-tls\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.437195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.437061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.437195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.437082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fbb8f4e-55da-4da4-a6a6-34414e099976-metrics-client-ca\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.437195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.437123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.437195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.437151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tstll\" (UniqueName: \"kubernetes.io/projected/0fbb8f4e-55da-4da4-a6a6-34414e099976-kube-api-access-tstll\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.437195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.437174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.438086 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.438050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fbb8f4e-55da-4da4-a6a6-34414e099976-metrics-client-ca\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.440242 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.440188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.440355 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.440325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.440568 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.440511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.440818 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.440794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-grpc-tls\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.440985 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.440959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.441094 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.440960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0fbb8f4e-55da-4da4-a6a6-34414e099976-secret-thanos-querier-tls\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.454663 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.454638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tstll\" (UniqueName: \"kubernetes.io/projected/0fbb8f4e-55da-4da4-a6a6-34414e099976-kube-api-access-tstll\") pod \"thanos-querier-59b9f95cf-9pfx6\" (UID: \"0fbb8f4e-55da-4da4-a6a6-34414e099976\") " pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:23.563849 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:23.563756    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:24.762722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.762685    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-64d5b566b6-v6d9q"]
Jul 14 19:42:24.765342 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.765312    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.769151 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.769116    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7ldp1toua1t8k\""
Jul 14 19:42:24.769151 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.769142    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jtsq9\""
Jul 14 19:42:24.769393 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.769205    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 19:42:24.769393 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.769385    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 14 19:42:24.769521 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.769395    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 14 19:42:24.769521 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.769475    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 14 19:42:24.782040 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.781970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64d5b566b6-v6d9q"]
Jul 14 19:42:24.848341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.847958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e74e9c3f-051c-4a53-b998-e95da94f45c0-audit-log\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.848341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.848047    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cb5q\" (UniqueName: \"kubernetes.io/projected/e74e9c3f-051c-4a53-b998-e95da94f45c0-kube-api-access-8cb5q\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.848341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.848099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-secret-metrics-server-tls\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.848341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.848129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e74e9c3f-051c-4a53-b998-e95da94f45c0-metrics-server-audit-profiles\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.848341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.848172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74e9c3f-051c-4a53-b998-e95da94f45c0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.848341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.848193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-client-ca-bundle\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.848341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.848273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-secret-metrics-server-client-certs\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.948867 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.948824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-secret-metrics-server-client-certs\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.949054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.948896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e74e9c3f-051c-4a53-b998-e95da94f45c0-audit-log\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.949054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.948954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cb5q\" (UniqueName: \"kubernetes.io/projected/e74e9c3f-051c-4a53-b998-e95da94f45c0-kube-api-access-8cb5q\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.949054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.949013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-secret-metrics-server-tls\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.949054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.949042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e74e9c3f-051c-4a53-b998-e95da94f45c0-metrics-server-audit-profiles\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.949248 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.949222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74e9c3f-051c-4a53-b998-e95da94f45c0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.949308 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.949263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-client-ca-bundle\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.949446 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.949388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e74e9c3f-051c-4a53-b998-e95da94f45c0-audit-log\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.950109 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.950079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74e9c3f-051c-4a53-b998-e95da94f45c0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.950324 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.950300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e74e9c3f-051c-4a53-b998-e95da94f45c0-metrics-server-audit-profiles\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.952069 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.952043    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-secret-metrics-server-tls\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.952183 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.952095    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-client-ca-bundle\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.952753 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.952732    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e74e9c3f-051c-4a53-b998-e95da94f45c0-secret-metrics-server-client-certs\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:24.986224 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:24.986187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cb5q\" (UniqueName: \"kubernetes.io/projected/e74e9c3f-051c-4a53-b998-e95da94f45c0-kube-api-access-8cb5q\") pod \"metrics-server-64d5b566b6-v6d9q\" (UID: \"e74e9c3f-051c-4a53-b998-e95da94f45c0\") " pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:25.078694 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:25.078589    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:25.822072 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:25.822036    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99cd6dd_2793_41b4_87b3_bd56e4450401.slice/crio-52fefc2822ed92049b881397847eb7d88c4045d48346014b729bae1889731176 WatchSource:0}: Error finding container 52fefc2822ed92049b881397847eb7d88c4045d48346014b729bae1889731176: Status 404 returned error can't find the container with id 52fefc2822ed92049b881397847eb7d88c4045d48346014b729bae1889731176
Jul 14 19:42:25.986052 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:25.986025    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"]
Jul 14 19:42:25.987970 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:25.987938    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fbb8f4e_55da_4da4_a6a6_34414e099976.slice/crio-0e77d96034ba3624352e5804dd3a47a5e4dc2245a429e2143a53e1add491ef02 WatchSource:0}: Error finding container 0e77d96034ba3624352e5804dd3a47a5e4dc2245a429e2143a53e1add491ef02: Status 404 returned error can't find the container with id 0e77d96034ba3624352e5804dd3a47a5e4dc2245a429e2143a53e1add491ef02
Jul 14 19:42:26.087608 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.087501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-dgcpn" event={"ID":"24d3d98f-c435-40da-8683-3d266890b3ff","Type":"ContainerStarted","Data":"96f3eea1c014650bb93eb8cbd28a02b024bba829567ba893ecf5307135951df6"}
Jul 14 19:42:26.087785 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.087738    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-dgcpn"
Jul 14 19:42:26.089154 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.089107    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvn6s" event={"ID":"f99cd6dd-2793-41b4-87b3-bd56e4450401","Type":"ContainerStarted","Data":"52fefc2822ed92049b881397847eb7d88c4045d48346014b729bae1889731176"}
Jul 14 19:42:26.089487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.089456    2581 patch_prober.go:28] interesting pod/downloads-68c9cbb497-dgcpn container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.18:8080/\": dial tcp 10.134.0.18:8080: connect: connection refused" start-of-body=
Jul 14 19:42:26.089615 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.089531    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-dgcpn" podUID="24d3d98f-c435-40da-8683-3d266890b3ff" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.18:8080/\": dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 14 19:42:26.090609 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.090579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" event={"ID":"0fbb8f4e-55da-4da4-a6a6-34414e099976","Type":"ContainerStarted","Data":"0e77d96034ba3624352e5804dd3a47a5e4dc2245a429e2143a53e1add491ef02"}
Jul 14 19:42:26.106550 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.106495    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-dgcpn" podStartSLOduration=1.293265243 podStartE2EDuration="19.106474059s" totalImagesPullingTime="17.813208816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:26.105311448 +0000 UTC m=+244.417010907" watchObservedRunningTime="2026-07-14 19:42:26.106474059 +0000 UTC m=+244.418173515"
Jul 14 19:42:26.199487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.199379    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64d5b566b6-v6d9q"]
Jul 14 19:42:26.201792 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.201675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-vpkrs"]
Jul 14 19:42:26.202477 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:26.202334    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74e9c3f_051c_4a53_b998_e95da94f45c0.slice/crio-12bea054477b3ba46193b6eefa09cfae53b7bcf557ec0242266788a258a1cb5b WatchSource:0}: Error finding container 12bea054477b3ba46193b6eefa09cfae53b7bcf557ec0242266788a258a1cb5b: Status 404 returned error can't find the container with id 12bea054477b3ba46193b6eefa09cfae53b7bcf557ec0242266788a258a1cb5b
Jul 14 19:42:26.521099 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.520150    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b67bd46df-nt5fl"]
Jul 14 19:42:26.523216 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.522720    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.528993 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.528381    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 14 19:42:26.528993 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.528798    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 14 19:42:26.529843 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.529658    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 14 19:42:26.529974 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.529869    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 14 19:42:26.530575 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.530074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5tp2h\""
Jul 14 19:42:26.532161 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.531889    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 14 19:42:26.535541 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.534530    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 14 19:42:26.538961 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.538474    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b67bd46df-nt5fl"]
Jul 14 19:42:26.561659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.561233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-service-ca\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.561659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.561301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-trusted-ca-bundle\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.561659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.561334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxpgb\" (UniqueName: \"kubernetes.io/projected/dec0116d-4f19-482a-ab50-5661e80a1618-kube-api-access-kxpgb\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.561659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.561376    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-oauth-serving-cert\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.561659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.561434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-console-config\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.561659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.561462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-serving-cert\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.561659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.561503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-oauth-config\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.663791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.662155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-oauth-serving-cert\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.663791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.662216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-console-config\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.663791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.662250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-serving-cert\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.663791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.662303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-oauth-config\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.663791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.662385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-service-ca\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.663791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.662449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-trusted-ca-bundle\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.663791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.662473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxpgb\" (UniqueName: \"kubernetes.io/projected/dec0116d-4f19-482a-ab50-5661e80a1618-kube-api-access-kxpgb\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.667005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.664995    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-service-ca\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.667005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.665668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-console-config\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.667005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.666048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-oauth-serving-cert\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.667005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.666945    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-trusted-ca-bundle\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.674749 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.674721    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-oauth-config\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.677790 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.677734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-serving-cert\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.677951 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.677823    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxpgb\" (UniqueName: \"kubernetes.io/projected/dec0116d-4f19-482a-ab50-5661e80a1618-kube-api-access-kxpgb\") pod \"console-7b67bd46df-nt5fl\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") " pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:26.842930 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:26.842583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:27.099040 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:27.098913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q" event={"ID":"e74e9c3f-051c-4a53-b998-e95da94f45c0","Type":"ContainerStarted","Data":"12bea054477b3ba46193b6eefa09cfae53b7bcf557ec0242266788a258a1cb5b"}
Jul 14 19:42:27.102350 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:27.102108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs" event={"ID":"e0870fac-f247-4234-be87-42613c05d2a2","Type":"ContainerStarted","Data":"88095d1097ddcff868159e093e83d2cb40c2250e1b29c7cb06fe00b7afc95be6"}
Jul 14 19:42:27.113394 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:27.113344    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-dgcpn"
Jul 14 19:42:27.429328 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:27.429236    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b67bd46df-nt5fl"]
Jul 14 19:42:27.591909 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:27.591450    2581 patch_prober.go:28] interesting pod/image-registry-57656676ff-x95kl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:27.591909 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:27.591517    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-57656676ff-x95kl" podUID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:28.034218 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:28.034162    2581 patch_prober.go:28] interesting pod/image-registry-676f444bdd-6wthl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:28.035041 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:28.034747    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:28.108372 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:28.108316    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b67bd46df-nt5fl" event={"ID":"dec0116d-4f19-482a-ab50-5661e80a1618","Type":"ContainerStarted","Data":"b0520a89a88077a140f0b52efb243b2f4308aa460714724e75dae9c9e66b1a7c"}
Jul 14 19:42:28.111970 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:28.111907    2581 generic.go:356] "Generic (PLEG): container finished" podID="f99cd6dd-2793-41b4-87b3-bd56e4450401" containerID="73b7a7c7ac2b94008a6f2c8e8cb6871e7856d149f1732fbd4c3dd688bcb90a29" exitCode=0
Jul 14 19:42:28.112161 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:28.112030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvn6s" event={"ID":"f99cd6dd-2793-41b4-87b3-bd56e4450401","Type":"ContainerDied","Data":"73b7a7c7ac2b94008a6f2c8e8cb6871e7856d149f1732fbd4c3dd688bcb90a29"}
Jul 14 19:42:30.024480 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:30.024418    2581 patch_prober.go:28] interesting pod/image-registry-676f444bdd-6wthl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:30.024956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:30.024563    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:31.132751 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.132706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs" event={"ID":"e0870fac-f247-4234-be87-42613c05d2a2","Type":"ContainerStarted","Data":"9e11c53de98f4b2f46185c1bd67d2f25ed3fad64b852352ecb051c3b0608fbde"}
Jul 14 19:42:31.133450 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.132793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs" event={"ID":"e0870fac-f247-4234-be87-42613c05d2a2","Type":"ContainerStarted","Data":"645944dfc639495fd03e3184a2e33b443281b05a44525ca58021da7185658022"}
Jul 14 19:42:31.133450 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.132830    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs" event={"ID":"e0870fac-f247-4234-be87-42613c05d2a2","Type":"ContainerStarted","Data":"8bd0801aba82997912ae42672ec06aeb524d5d8dba5627d6c5f169379a09501d"}
Jul 14 19:42:31.136056 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.135877    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" event={"ID":"0fbb8f4e-55da-4da4-a6a6-34414e099976","Type":"ContainerStarted","Data":"8ea1d5bf0e89bc3b2117b300b4d36fe96439c8663c009a36ac8fa45e16b78013"}
Jul 14 19:42:31.136056 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.135956    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" event={"ID":"0fbb8f4e-55da-4da4-a6a6-34414e099976","Type":"ContainerStarted","Data":"d5e0fa2a3edde433235fecfd28d6abf4d5629450815b8bb6751db3e128fd744a"}
Jul 14 19:42:31.136056 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.136000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" event={"ID":"0fbb8f4e-55da-4da4-a6a6-34414e099976","Type":"ContainerStarted","Data":"b9bc2c6235c40f12dd20f0a0120f1005851fc6dbfccd9e93cacee5558284cd69"}
Jul 14 19:42:31.138292 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.138235    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q" event={"ID":"e74e9c3f-051c-4a53-b998-e95da94f45c0","Type":"ContainerStarted","Data":"b08aa7239064ee70136be95b9a598b1745048f1f021684ecec0b2fc7b25ca3ca"}
Jul 14 19:42:31.141425 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.141375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvn6s" event={"ID":"f99cd6dd-2793-41b4-87b3-bd56e4450401","Type":"ContainerStarted","Data":"ea7a611fd91bcb2ddb5ca03c4b80a30c8515feca342253bec41bd85899e3400d"}
Jul 14 19:42:31.141591 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.141458    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvn6s" event={"ID":"f99cd6dd-2793-41b4-87b3-bd56e4450401","Type":"ContainerStarted","Data":"d66cf3d96a1d1b1b4b5235fb4a3a1083fac019e5424b0c43ce70d56c02b661d8"}
Jul 14 19:42:31.166071 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.164847    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-vpkrs" podStartSLOduration=7.315441872 podStartE2EDuration="11.164821602s" totalImagesPullingTime="3.84937973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:31.161965409 +0000 UTC m=+249.473664876" watchObservedRunningTime="2026-07-14 19:42:31.164821602 +0000 UTC m=+249.476521059"
Jul 14 19:42:31.195383 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.195324    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qvn6s" podStartSLOduration=9.673770255 podStartE2EDuration="11.195306242s" totalImagesPullingTime="1.417691125s" totalInitContainerRuntime="103.844862ms" isStatefulPod=true podCreationTimestamp="2026-07-14 19:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:31.19463369 +0000 UTC m=+249.506333146" watchObservedRunningTime="2026-07-14 19:42:31.195306242 +0000 UTC m=+249.507005698"
Jul 14 19:42:31.218954 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:31.218884    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q" podStartSLOduration=3.363187198 podStartE2EDuration="7.218849065s" totalImagesPullingTime="3.855661867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:31.217301737 +0000 UTC m=+249.529001192" watchObservedRunningTime="2026-07-14 19:42:31.218849065 +0000 UTC m=+249.530548528"
Jul 14 19:42:32.613652 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:32.613358    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57656676ff-x95kl" podUID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" containerName="registry" containerID="cri-o://3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324" gracePeriod=30
Jul 14 19:42:33.017637 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.017596    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:42:33.041364 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041328    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-certificates\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041364 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041379    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-trusted-ca\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr6h8\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-kube-api-access-jr6h8\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041519    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-image-registry-private-configuration\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041544    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041604    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-bound-sa-token\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041821 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-ca-trust-extracted\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041821 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041664    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-installation-pull-secrets\") pod \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\" (UID: \"52f9adc9-9ebe-4728-bf7d-4e20a11887bb\") "
Jul 14 19:42:33.041931 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041804    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:42:33.041931 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041845    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:42:33.042054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.041998    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-certificates\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.042054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.042021    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-trusted-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.044553 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.044525    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:42:33.046057 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.046010    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-kube-api-access-jr6h8" (OuterVolumeSpecName: "kube-api-access-jr6h8") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "kube-api-access-jr6h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:42:33.046299 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.046267    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:42:33.047329 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.047299    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:42:33.047673 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.047645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:42:33.054024 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.053978    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "52f9adc9-9ebe-4728-bf7d-4e20a11887bb" (UID: "52f9adc9-9ebe-4728-bf7d-4e20a11887bb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:42:33.143467 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.143427    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-image-registry-private-configuration\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.143467 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.143462    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-registry-tls\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.143694 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.143479    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-bound-sa-token\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.143694 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.143493    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-ca-trust-extracted\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.143694 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.143504    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-installation-pull-secrets\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.143694 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.143514    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr6h8\" (UniqueName: \"kubernetes.io/projected/52f9adc9-9ebe-4728-bf7d-4e20a11887bb-kube-api-access-jr6h8\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:42:33.152752 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.152710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b67bd46df-nt5fl" event={"ID":"dec0116d-4f19-482a-ab50-5661e80a1618","Type":"ContainerStarted","Data":"8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3"}
Jul 14 19:42:33.154260 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.154227    2581 generic.go:356] "Generic (PLEG): container finished" podID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" containerID="3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324" exitCode=0
Jul 14 19:42:33.154260 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.154261    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57656676ff-x95kl"
Jul 14 19:42:33.154473 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.154306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57656676ff-x95kl" event={"ID":"52f9adc9-9ebe-4728-bf7d-4e20a11887bb","Type":"ContainerDied","Data":"3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324"}
Jul 14 19:42:33.154473 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.154353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57656676ff-x95kl" event={"ID":"52f9adc9-9ebe-4728-bf7d-4e20a11887bb","Type":"ContainerDied","Data":"8aeddc55a9d0051b31a8a93b6b26552fe13165eeb62168bcff833b0bf78e32e3"}
Jul 14 19:42:33.154473 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.154376    2581 scope.go:117] "RemoveContainer" containerID="3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324"
Jul 14 19:42:33.164447 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.164387    2581 scope.go:117] "RemoveContainer" containerID="3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324"
Jul 14 19:42:33.164815 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:42:33.164781    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324\": container with ID starting with 3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324 not found: ID does not exist" containerID="3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324"
Jul 14 19:42:33.164898 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.164826    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324"} err="failed to get container status \"3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324\": rpc error: code = NotFound desc = could not find container \"3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324\": container with ID starting with 3640e0706f2aa06433db07b2e2ee6b86fa9b2d6fd21a80010001bc143cf32324 not found: ID does not exist"
Jul 14 19:42:33.198179 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.197784    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b67bd46df-nt5fl" podStartSLOduration=2.036918951 podStartE2EDuration="7.197763097s" totalImagesPullingTime="5.160844146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:33.197762661 +0000 UTC m=+251.509462118" watchObservedRunningTime="2026-07-14 19:42:33.197763097 +0000 UTC m=+251.509462553"
Jul 14 19:42:33.219569 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.219525    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57656676ff-x95kl"]
Jul 14 19:42:33.224244 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.224203    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57656676ff-x95kl"]
Jul 14 19:42:33.340794 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:33.340714    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" path="/var/lib/kubelet/pods/52f9adc9-9ebe-4728-bf7d-4e20a11887bb/volumes"
Jul 14 19:42:34.163318 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:34.163275    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" event={"ID":"0fbb8f4e-55da-4da4-a6a6-34414e099976","Type":"ContainerStarted","Data":"b63f60ab527379d275d83823c85dfe71c4c216189668e861782e93590e4037ce"}
Jul 14 19:42:34.163877 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:34.163371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" event={"ID":"0fbb8f4e-55da-4da4-a6a6-34414e099976","Type":"ContainerStarted","Data":"46cac949c884452a346f8c36472c6d671dd5836eb5586cca60c8a1ac3d9cfd76"}
Jul 14 19:42:34.163877 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:34.163439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" event={"ID":"0fbb8f4e-55da-4da4-a6a6-34414e099976","Type":"ContainerStarted","Data":"3579cd0fa86902d730ca71e7de99f99543ab7dd8dcdab3448183fdd1ff5fc4da"}
Jul 14 19:42:34.197580 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:34.197510    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6" podStartSLOduration=4.367278698 podStartE2EDuration="11.197484838s" totalImagesPullingTime="6.83020614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:42:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:34.19736884 +0000 UTC m=+252.509068331" watchObservedRunningTime="2026-07-14 19:42:34.197484838 +0000 UTC m=+252.509184294"
Jul 14 19:42:35.164574 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:35.164534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:42:35.166973 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:35.166928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:35.167117 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:35.167084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24cb2861-5df6-4718-9b15-38f935fa2397-metrics-certs\") pod \"network-metrics-daemon-4jvmg\" (UID: \"24cb2861-5df6-4718-9b15-38f935fa2397\") " pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:42:35.172836 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:35.172812    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-59b9f95cf-9pfx6"
Jul 14 19:42:35.335355 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:35.335327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9cvvf\""
Jul 14 19:42:35.343675 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:35.343651    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvmg"
Jul 14 19:42:35.491210 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:35.491178    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4jvmg"]
Jul 14 19:42:35.494315 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:42:35.494282    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cb2861_5df6_4718_9b15_38f935fa2397.slice/crio-ad408d8dd1bdc246a8e4ce5b956038f3595e53582797450558684481eb07e5f1 WatchSource:0}: Error finding container ad408d8dd1bdc246a8e4ce5b956038f3595e53582797450558684481eb07e5f1: Status 404 returned error can't find the container with id ad408d8dd1bdc246a8e4ce5b956038f3595e53582797450558684481eb07e5f1
Jul 14 19:42:36.172832 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:36.172762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jvmg" event={"ID":"24cb2861-5df6-4718-9b15-38f935fa2397","Type":"ContainerStarted","Data":"ad408d8dd1bdc246a8e4ce5b956038f3595e53582797450558684481eb07e5f1"}
Jul 14 19:42:36.843203 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:36.843159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:36.843442 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:36.843237    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:42:36.844715 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:36.844688    2581 patch_prober.go:28] interesting pod/console-7b67bd46df-nt5fl container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.23:8443/health\": dial tcp 10.134.0.23:8443: connect: connection refused" start-of-body=
Jul 14 19:42:36.844830 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:36.844755    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b67bd46df-nt5fl" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" containerName="console" probeResult="failure" output="Get \"https://10.134.0.23:8443/health\": dial tcp 10.134.0.23:8443: connect: connection refused"
Jul 14 19:42:37.178746 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:37.178711    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jvmg" event={"ID":"24cb2861-5df6-4718-9b15-38f935fa2397","Type":"ContainerStarted","Data":"6ac6048c9ea926101238fae1ec39c3fc10a0b9c77f89178b5b8fbe8f224d47ce"}
Jul 14 19:42:38.032543 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:38.032507    2581 patch_prober.go:28] interesting pod/image-registry-676f444bdd-6wthl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:38.032825 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:38.032568    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:38.184465 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:38.184430    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jvmg" event={"ID":"24cb2861-5df6-4718-9b15-38f935fa2397","Type":"ContainerStarted","Data":"313e83ebab6d939c90e11eea73836944856f5feb4d05dc9bb4aaee6e9b01ed53"}
Jul 14 19:42:38.203836 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:38.203774    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4jvmg" podStartSLOduration=253.782257446 podStartE2EDuration="4m15.203745419s" totalImagesPullingTime="1.421487973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:42:38.20304634 +0000 UTC m=+256.514745798" watchObservedRunningTime="2026-07-14 19:42:38.203745419 +0000 UTC m=+256.515444876"
Jul 14 19:42:40.023438 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:40.023381    2581 patch_prober.go:28] interesting pod/image-registry-676f444bdd-6wthl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:40.023831 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:40.023458    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:45.079292 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:45.079199    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:45.079292 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:45.079294    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:42:46.843937 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:46.843897    2581 patch_prober.go:28] interesting pod/console-7b67bd46df-nt5fl container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.23:8443/health\": dial tcp 10.134.0.23:8443: connect: connection refused" start-of-body=
Jul 14 19:42:46.844343 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:46.843957    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b67bd46df-nt5fl" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" containerName="console" probeResult="failure" output="Get \"https://10.134.0.23:8443/health\": dial tcp 10.134.0.23:8443: connect: connection refused"
Jul 14 19:42:48.032244 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:48.032209    2581 patch_prober.go:28] interesting pod/image-registry-676f444bdd-6wthl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:48.032669 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:48.032272    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:48.032669 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:48.032317    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:42:48.032907 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:48.032880    2581 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"97c19326423aa00be58f658c913718c30a4f1d4d7e98b354338478e83f7f83cb"} pod="openshift-image-registry/image-registry-676f444bdd-6wthl" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 14 19:42:48.036422 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:48.036374    2581 patch_prober.go:28] interesting pod/image-registry-676f444bdd-6wthl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:48.036548 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:48.036437    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:52.246654 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:52.246613    2581 generic.go:356] "Generic (PLEG): container finished" podID="ad53b9d1-10bc-47f5-9544-f69c83ec3e02" containerID="b9e1b992fde193578c3671fa02d4a72abb1779da1cba2a6b7fe84cc585304a36" exitCode=0
Jul 14 19:42:52.246654 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:52.246655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n" event={"ID":"ad53b9d1-10bc-47f5-9544-f69c83ec3e02","Type":"ContainerDied","Data":"b9e1b992fde193578c3671fa02d4a72abb1779da1cba2a6b7fe84cc585304a36"}
Jul 14 19:42:52.247189 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:52.246971    2581 scope.go:117] "RemoveContainer" containerID="b9e1b992fde193578c3671fa02d4a72abb1779da1cba2a6b7fe84cc585304a36"
Jul 14 19:42:53.252788 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:53.252751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-nqs9n" event={"ID":"ad53b9d1-10bc-47f5-9544-f69c83ec3e02","Type":"ContainerStarted","Data":"b6be98b74433b1cbf83ccc34ce2e160b839e4d782058103a5abad24aa8dde82d"}
Jul 14 19:42:56.115105 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:56.115074    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-222tg_79b168f0-d438-4064-9ee3-97bad5dfebfd/cluster-monitoring-operator/0.log"
Jul 14 19:42:56.315125 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:56.315095    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vpkrs_e0870fac-f247-4234-be87-42613c05d2a2/kube-state-metrics/0.log"
Jul 14 19:42:56.515748 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:56.515717    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vpkrs_e0870fac-f247-4234-be87-42613c05d2a2/kube-rbac-proxy-main/0.log"
Jul 14 19:42:56.714645 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:56.714617    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vpkrs_e0870fac-f247-4234-be87-42613c05d2a2/kube-rbac-proxy-self/0.log"
Jul 14 19:42:56.843311 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:56.843219    2581 patch_prober.go:28] interesting pod/console-7b67bd46df-nt5fl container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.23:8443/health\": dial tcp 10.134.0.23:8443: connect: connection refused" start-of-body=
Jul 14 19:42:56.843311 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:56.843284    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b67bd46df-nt5fl" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" containerName="console" probeResult="failure" output="Get \"https://10.134.0.23:8443/health\": dial tcp 10.134.0.23:8443: connect: connection refused"
Jul 14 19:42:56.917969 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:56.917936    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64d5b566b6-v6d9q_e74e9c3f-051c-4a53-b998-e95da94f45c0/metrics-server/0.log"
Jul 14 19:42:57.913478 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:57.913450    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvn6s_f99cd6dd-2793-41b4-87b3-bd56e4450401/init-textfile/0.log"
Jul 14 19:42:58.037173 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:58.037139    2581 patch_prober.go:28] interesting pod/image-registry-676f444bdd-6wthl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 19:42:58.037330 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:58.037195    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:42:58.115134 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:58.115105    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvn6s_f99cd6dd-2793-41b4-87b3-bd56e4450401/node-exporter/0.log"
Jul 14 19:42:58.313992 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:42:58.313960    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvn6s_f99cd6dd-2793-41b4-87b3-bd56e4450401/kube-rbac-proxy/0.log"
Jul 14 19:43:01.715915 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:01.715886    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/thanos-query/0.log"
Jul 14 19:43:01.914049 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:01.914016    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy-web/0.log"
Jul 14 19:43:02.114524 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:02.114438    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy/0.log"
Jul 14 19:43:02.313577 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:02.313548    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/prom-label-proxy/0.log"
Jul 14 19:43:02.516005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:02.515972    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy-rules/0.log"
Jul 14 19:43:02.716313 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:02.716279    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy-metrics/0.log"
Jul 14 19:43:02.914693 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:02.914612    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-q68tv_de28e053-ed08-45f7-acd3-986ae8acd2fe/networking-console-plugin/0.log"
Jul 14 19:43:03.115520 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:03.115486    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:43:03.317212 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:03.317185    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/2.log"
Jul 14 19:43:03.713830 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:03.713800    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b67bd46df-nt5fl_dec0116d-4f19-482a-ab50-5661e80a1618/console/0.log"
Jul 14 19:43:04.116007 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:04.115935    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-dgcpn_24d3d98f-c435-40da-8683-3d266890b3ff/download-server/0.log"
Jul 14 19:43:05.085106 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:05.085071    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:43:05.089500 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:05.089471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-64d5b566b6-v6d9q"
Jul 14 19:43:05.914049 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:05.914016    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z826t_cc6571ea-7b0d-4671-a89b-e427c40cecc4/dns-node-resolver/0.log"
Jul 14 19:43:06.314641 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:06.314609    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-676f444bdd-6wthl_dd6c5539-7faf-4701-b16b-237a56f2ef5e/registry/0.log"
Jul 14 19:43:06.514626 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:06.514596    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5p8x5_13c49190-4865-4f90-a7f6-3400303c9879/node-ca/0.log"
Jul 14 19:43:06.847353 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:06.847321    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:43:06.851165 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:06.851128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:43:07.517629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:07.517597    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tpdjw_99fa2d76-352e-406b-84e4-968d0ffd15da/serve-healthcheck-canary/0.log"
Jul 14 19:43:07.786964 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:43:07.786858    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nn5jk" podUID="0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f"
Jul 14 19:43:08.038021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:08.037946    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:43:08.313756 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:08.313671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:43:11.088468 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:11.088380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:43:11.090803 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:11.090779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f-metrics-tls\") pod \"dns-default-nn5jk\" (UID: \"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f\") " pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:43:11.317079 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:11.317047    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-774zx\""
Jul 14 19:43:11.324940 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:11.324899    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:43:11.574203 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:11.574116    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nn5jk"]
Jul 14 19:43:11.577368 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:43:11.577330    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e87ee6b_a59f_41b0_b2d1_6a60fa45c81f.slice/crio-9f8c6aeb1a0b248e5a8e5bbf6bbe8585dd094fa7decf254787252ec324f1479b WatchSource:0}: Error finding container 9f8c6aeb1a0b248e5a8e5bbf6bbe8585dd094fa7decf254787252ec324f1479b: Status 404 returned error can't find the container with id 9f8c6aeb1a0b248e5a8e5bbf6bbe8585dd094fa7decf254787252ec324f1479b
Jul 14 19:43:12.337067 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:12.337024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn5jk" event={"ID":"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f","Type":"ContainerStarted","Data":"9f8c6aeb1a0b248e5a8e5bbf6bbe8585dd094fa7decf254787252ec324f1479b"}
Jul 14 19:43:13.052808 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:13.052755    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" podUID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerName="registry" containerID="cri-o://97c19326423aa00be58f658c913718c30a4f1d4d7e98b354338478e83f7f83cb" gracePeriod=30
Jul 14 19:43:14.348637 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.348596    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd6c5539-7faf-4701-b16b-237a56f2ef5e" containerID="97c19326423aa00be58f658c913718c30a4f1d4d7e98b354338478e83f7f83cb" exitCode=0
Jul 14 19:43:14.349162 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.348665    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" event={"ID":"dd6c5539-7faf-4701-b16b-237a56f2ef5e","Type":"ContainerDied","Data":"97c19326423aa00be58f658c913718c30a4f1d4d7e98b354338478e83f7f83cb"}
Jul 14 19:43:14.349162 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.348736    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:43:14.349162 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.348754    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676f444bdd-6wthl" event={"ID":"dd6c5539-7faf-4701-b16b-237a56f2ef5e","Type":"ContainerStarted","Data":"55dfa3f12e9553c85b1e6e3be67f647708338baf1c36f8068ce4599b286acc87"}
Jul 14 19:43:14.350657 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.350633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn5jk" event={"ID":"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f","Type":"ContainerStarted","Data":"4372c9fb5017299fbd9a5d26f365a04d921b0f941b2cd823e6eb43e7af4d2ce2"}
Jul 14 19:43:14.350803 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.350680    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:43:14.350803 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.350698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn5jk" event={"ID":"0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f","Type":"ContainerStarted","Data":"49e6ca8bf67e13d1188ddb0a65f7c8894fc8564c808a418723de102f2a8ac77e"}
Jul 14 19:43:14.394610 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:14.394481    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nn5jk" podStartSLOduration=251.29340014 podStartE2EDuration="4m13.394454677s" totalImagesPullingTime="2.101054537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:43:14.392488829 +0000 UTC m=+292.704188287" watchObservedRunningTime="2026-07-14 19:43:14.394454677 +0000 UTC m=+292.706154134"
Jul 14 19:43:23.160690 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:23.160659    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:43:23.164341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:23.164311    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:43:23.178589 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:23.178561    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 19:43:24.357580 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:24.357548    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nn5jk"
Jul 14 19:43:35.359552 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:43:35.359518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-676f444bdd-6wthl"
Jul 14 19:44:02.370940 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.370898    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b67bd46df-nt5fl"]
Jul 14 19:44:02.491054 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.491009    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f5594c478-qklfc"]
Jul 14 19:44:02.491369 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.491353    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" containerName="registry"
Jul 14 19:44:02.491369 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.491369    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" containerName="registry"
Jul 14 19:44:02.491526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.491448    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f9adc9-9ebe-4728-bf7d-4e20a11887bb" containerName="registry"
Jul 14 19:44:02.494877 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.494850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.505607 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.505580    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5594c478-qklfc"]
Jul 14 19:44:02.531733 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.531690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-serving-cert\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.531733 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.531729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps8sc\" (UniqueName: \"kubernetes.io/projected/ab6ebd30-9871-462f-a06c-0bd3b1e25452-kube-api-access-ps8sc\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.531956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.531761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-oauth-config\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.531956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.531802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-oauth-serving-cert\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.531956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.531844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-service-ca\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.531956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.531901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-config\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.531956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.531926    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-trusted-ca-bundle\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.633461 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.633361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-oauth-config\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.633461 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.633423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-oauth-serving-cert\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.633461 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.633454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-service-ca\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.633705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.633488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-config\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.633705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.633508    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-trusted-ca-bundle\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.633705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.633567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-serving-cert\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.633705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.633601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps8sc\" (UniqueName: \"kubernetes.io/projected/ab6ebd30-9871-462f-a06c-0bd3b1e25452-kube-api-access-ps8sc\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.634282 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.634256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-service-ca\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.634429 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.634325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-oauth-serving-cert\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.634429 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.634339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-config\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.634513 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.634491    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-trusted-ca-bundle\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.636065 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.636024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-oauth-config\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.636195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.636120    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-serving-cert\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.642670 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.642638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps8sc\" (UniqueName: \"kubernetes.io/projected/ab6ebd30-9871-462f-a06c-0bd3b1e25452-kube-api-access-ps8sc\") pod \"console-7f5594c478-qklfc\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") " pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.805629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.805586    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:02.948966 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:02.948929    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5594c478-qklfc"]
Jul 14 19:44:02.952268 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:44:02.952235    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6ebd30_9871_462f_a06c_0bd3b1e25452.slice/crio-bef129a68a09ab6606aebbd18217aa41ffbd27fc6edfb97ef46ba965da33b15e WatchSource:0}: Error finding container bef129a68a09ab6606aebbd18217aa41ffbd27fc6edfb97ef46ba965da33b15e: Status 404 returned error can't find the container with id bef129a68a09ab6606aebbd18217aa41ffbd27fc6edfb97ef46ba965da33b15e
Jul 14 19:44:03.559147 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:03.559109    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5594c478-qklfc" event={"ID":"ab6ebd30-9871-462f-a06c-0bd3b1e25452","Type":"ContainerStarted","Data":"c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe"}
Jul 14 19:44:03.559718 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:03.559164    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5594c478-qklfc" event={"ID":"ab6ebd30-9871-462f-a06c-0bd3b1e25452","Type":"ContainerStarted","Data":"bef129a68a09ab6606aebbd18217aa41ffbd27fc6edfb97ef46ba965da33b15e"}
Jul 14 19:44:03.583459 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:03.580580    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f5594c478-qklfc" podStartSLOduration=1.580557345 podStartE2EDuration="1.580557345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:44:03.578141181 +0000 UTC m=+341.889840633" watchObservedRunningTime="2026-07-14 19:44:03.580557345 +0000 UTC m=+341.892256801"
Jul 14 19:44:12.806329 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:12.806272    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:12.806849 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:12.806350    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:12.811318 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:12.811292    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:13.602058 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:13.601974    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:44:27.392863 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.392791    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b67bd46df-nt5fl" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" containerName="console" containerID="cri-o://8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3" gracePeriod=15
Jul 14 19:44:27.653978 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.653907    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:44:27.655016 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.654989    2581 generic.go:356] "Generic (PLEG): container finished" podID="dec0116d-4f19-482a-ab50-5661e80a1618" containerID="8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3" exitCode=0
Jul 14 19:44:27.655128 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.655057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b67bd46df-nt5fl" event={"ID":"dec0116d-4f19-482a-ab50-5661e80a1618","Type":"ContainerDied","Data":"8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3"}
Jul 14 19:44:27.655128 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.655106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b67bd46df-nt5fl" event={"ID":"dec0116d-4f19-482a-ab50-5661e80a1618","Type":"ContainerDied","Data":"b0520a89a88077a140f0b52efb243b2f4308aa460714724e75dae9c9e66b1a7c"}
Jul 14 19:44:27.655196 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.655130    2581 scope.go:117] "RemoveContainer" containerID="8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3"
Jul 14 19:44:27.663004 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.662987    2581 scope.go:117] "RemoveContainer" containerID="8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3"
Jul 14 19:44:27.663299 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:44:27.663273    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3\": container with ID starting with 8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3 not found: ID does not exist" containerID="8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3"
Jul 14 19:44:27.663372 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.663310    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3"} err="failed to get container status \"8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3\": rpc error: code = NotFound desc = could not find container \"8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3\": container with ID starting with 8e63b2a9150cd2638c07b16eff187a4d5c19885b34d45c12e55f4d0fe8ac59f3 not found: ID does not exist"
Jul 14 19:44:27.766992 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.766954    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-oauth-serving-cert\") pod \"dec0116d-4f19-482a-ab50-5661e80a1618\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") "
Jul 14 19:44:27.767201 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767022    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-trusted-ca-bundle\") pod \"dec0116d-4f19-482a-ab50-5661e80a1618\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") "
Jul 14 19:44:27.767201 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767043    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxpgb\" (UniqueName: \"kubernetes.io/projected/dec0116d-4f19-482a-ab50-5661e80a1618-kube-api-access-kxpgb\") pod \"dec0116d-4f19-482a-ab50-5661e80a1618\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") "
Jul 14 19:44:27.767201 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767078    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-console-config\") pod \"dec0116d-4f19-482a-ab50-5661e80a1618\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") "
Jul 14 19:44:27.767201 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767099    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-oauth-config\") pod \"dec0116d-4f19-482a-ab50-5661e80a1618\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") "
Jul 14 19:44:27.767201 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-service-ca\") pod \"dec0116d-4f19-482a-ab50-5661e80a1618\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") "
Jul 14 19:44:27.767201 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767173    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-serving-cert\") pod \"dec0116d-4f19-482a-ab50-5661e80a1618\" (UID: \"dec0116d-4f19-482a-ab50-5661e80a1618\") "
Jul 14 19:44:27.767550 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767484    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dec0116d-4f19-482a-ab50-5661e80a1618" (UID: "dec0116d-4f19-482a-ab50-5661e80a1618"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:44:27.767550 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dec0116d-4f19-482a-ab50-5661e80a1618" (UID: "dec0116d-4f19-482a-ab50-5661e80a1618"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:44:27.767630 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767542    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-console-config" (OuterVolumeSpecName: "console-config") pod "dec0116d-4f19-482a-ab50-5661e80a1618" (UID: "dec0116d-4f19-482a-ab50-5661e80a1618"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:44:27.767924 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.767897    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-service-ca" (OuterVolumeSpecName: "service-ca") pod "dec0116d-4f19-482a-ab50-5661e80a1618" (UID: "dec0116d-4f19-482a-ab50-5661e80a1618"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:44:27.769466 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.769435    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dec0116d-4f19-482a-ab50-5661e80a1618" (UID: "dec0116d-4f19-482a-ab50-5661e80a1618"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:44:27.769570 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.769496    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dec0116d-4f19-482a-ab50-5661e80a1618" (UID: "dec0116d-4f19-482a-ab50-5661e80a1618"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:44:27.769570 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.769561    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dec0116d-4f19-482a-ab50-5661e80a1618-kube-api-access-kxpgb" (OuterVolumeSpecName: "kube-api-access-kxpgb") pod "dec0116d-4f19-482a-ab50-5661e80a1618" (UID: "dec0116d-4f19-482a-ab50-5661e80a1618"). InnerVolumeSpecName "kube-api-access-kxpgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:44:27.868213 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.868171    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-serving-cert\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:44:27.868213 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.868205    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-oauth-serving-cert\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:44:27.868213 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.868214    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-trusted-ca-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:44:27.868213 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.868224    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxpgb\" (UniqueName: \"kubernetes.io/projected/dec0116d-4f19-482a-ab50-5661e80a1618-kube-api-access-kxpgb\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:44:27.868540 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.868234    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-console-config\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:44:27.868540 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.868242    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dec0116d-4f19-482a-ab50-5661e80a1618-console-oauth-config\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:44:27.868540 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:27.868250    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dec0116d-4f19-482a-ab50-5661e80a1618-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:44:28.660574 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:28.660545    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b67bd46df-nt5fl"
Jul 14 19:44:28.683221 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:28.683176    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b67bd46df-nt5fl"]
Jul 14 19:44:28.690876 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:28.690844    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b67bd46df-nt5fl"]
Jul 14 19:44:29.337075 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:29.337041    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" path="/var/lib/kubelet/pods/dec0116d-4f19-482a-ab50-5661e80a1618/volumes"
Jul 14 19:44:51.173467 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.173411    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"]
Jul 14 19:44:51.173912 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.173804    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" containerName="console"
Jul 14 19:44:51.173912 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.173817    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" containerName="console"
Jul 14 19:44:51.173912 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.173884    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dec0116d-4f19-482a-ab50-5661e80a1618" containerName="console"
Jul 14 19:44:51.176839 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.176821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.179958 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.179932    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 14 19:44:51.180063 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.179975    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5nshw\""
Jul 14 19:44:51.180494 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.180479    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 14 19:44:51.192699 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.192674    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"]
Jul 14 19:44:51.265344 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.265300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqtls\" (UniqueName: \"kubernetes.io/projected/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-kube-api-access-vqtls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.265581 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.265356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.265581 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.265439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.366655 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.366612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqtls\" (UniqueName: \"kubernetes.io/projected/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-kube-api-access-vqtls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.366863 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.366703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.366863 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.366762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.367136 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.367115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.367196 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.367147    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.376052 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.376017    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqtls\" (UniqueName: \"kubernetes.io/projected/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-kube-api-access-vqtls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.486342 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.486300    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:44:51.632610 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.632579    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"]
Jul 14 19:44:51.634990 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:44:51.634960    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46368b34_3fd2_4cb6_8132_8dbbc8feb1cf.slice/crio-0e657252eee35df65ba51bc31cd0d44a3b7960dc5cdb1b26dd4070a09ab59d32 WatchSource:0}: Error finding container 0e657252eee35df65ba51bc31cd0d44a3b7960dc5cdb1b26dd4070a09ab59d32: Status 404 returned error can't find the container with id 0e657252eee35df65ba51bc31cd0d44a3b7960dc5cdb1b26dd4070a09ab59d32
Jul 14 19:44:51.636941 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.636922    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 19:44:51.757488 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:51.757367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8" event={"ID":"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf","Type":"ContainerStarted","Data":"0e657252eee35df65ba51bc31cd0d44a3b7960dc5cdb1b26dd4070a09ab59d32"}
Jul 14 19:44:59.793995 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:59.793953    2581 generic.go:356] "Generic (PLEG): container finished" podID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerID="22be89c2052d9cba9ba8e873de84e17b79de48cf2c274a267a473be158ca43d7" exitCode=0
Jul 14 19:44:59.794446 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:44:59.794059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8" event={"ID":"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf","Type":"ContainerDied","Data":"22be89c2052d9cba9ba8e873de84e17b79de48cf2c274a267a473be158ca43d7"}
Jul 14 19:45:02.809334 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:02.809301    2581 generic.go:356] "Generic (PLEG): container finished" podID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerID="507bc63c5a68d24a344007a57d962865e23943af13e46fbf19d5fe610a74bb99" exitCode=0
Jul 14 19:45:02.809780 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:02.809349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8" event={"ID":"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf","Type":"ContainerDied","Data":"507bc63c5a68d24a344007a57d962865e23943af13e46fbf19d5fe610a74bb99"}
Jul 14 19:45:09.844985 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:09.844942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8" event={"ID":"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf","Type":"ContainerStarted","Data":"42b163ef4ecdddb05df2e3a9fc97f3e99a0b31bd37a69b516f9e906ffd8e9e26"}
Jul 14 19:45:09.866732 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:09.866680    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8" podStartSLOduration=2.064654763 podStartE2EDuration="18.866663606s" totalImagesPullingTime="16.802008843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:44:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:45:09.865112749 +0000 UTC m=+408.176812204" watchObservedRunningTime="2026-07-14 19:45:09.866663606 +0000 UTC m=+408.178363062"
Jul 14 19:45:10.850676 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:10.850638    2581 generic.go:356] "Generic (PLEG): container finished" podID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerID="42b163ef4ecdddb05df2e3a9fc97f3e99a0b31bd37a69b516f9e906ffd8e9e26" exitCode=0
Jul 14 19:45:10.851082 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:10.850726    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8" event={"ID":"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf","Type":"ContainerDied","Data":"42b163ef4ecdddb05df2e3a9fc97f3e99a0b31bd37a69b516f9e906ffd8e9e26"}
Jul 14 19:45:11.984115 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:11.984086    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:45:12.051775 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.051735    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqtls\" (UniqueName: \"kubernetes.io/projected/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-kube-api-access-vqtls\") pod \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") "
Jul 14 19:45:12.051976 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.051791    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-util\") pod \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") "
Jul 14 19:45:12.051976 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.051829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-bundle\") pod \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\" (UID: \"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf\") "
Jul 14 19:45:12.052526 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.052494    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-bundle" (OuterVolumeSpecName: "bundle") pod "46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" (UID: "46368b34-3fd2-4cb6-8132-8dbbc8feb1cf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:45:12.054069 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.054039    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-kube-api-access-vqtls" (OuterVolumeSpecName: "kube-api-access-vqtls") pod "46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" (UID: "46368b34-3fd2-4cb6-8132-8dbbc8feb1cf"). InnerVolumeSpecName "kube-api-access-vqtls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:45:12.056316 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.056288    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-util" (OuterVolumeSpecName: "util") pod "46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" (UID: "46368b34-3fd2-4cb6-8132-8dbbc8feb1cf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:45:12.153338 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.153240    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-util\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:12.153338 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.153272    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:12.153338 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.153285    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqtls\" (UniqueName: \"kubernetes.io/projected/46368b34-3fd2-4cb6-8132-8dbbc8feb1cf-kube-api-access-vqtls\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:12.861034 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.860990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8" event={"ID":"46368b34-3fd2-4cb6-8132-8dbbc8feb1cf","Type":"ContainerDied","Data":"0e657252eee35df65ba51bc31cd0d44a3b7960dc5cdb1b26dd4070a09ab59d32"}
Jul 14 19:45:12.861220 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.861045    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e657252eee35df65ba51bc31cd0d44a3b7960dc5cdb1b26dd4070a09ab59d32"
Jul 14 19:45:12.861220 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:12.861007    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404zjc8"
Jul 14 19:45:21.037776 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.037739    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"]
Jul 14 19:45:21.038279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.038077    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerName="util"
Jul 14 19:45:21.038279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.038089    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerName="util"
Jul 14 19:45:21.038279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.038097    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerName="pull"
Jul 14 19:45:21.038279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.038103    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerName="pull"
Jul 14 19:45:21.038279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.038118    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerName="extract"
Jul 14 19:45:21.038279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.038124    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerName="extract"
Jul 14 19:45:21.038279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.038172    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="46368b34-3fd2-4cb6-8132-8dbbc8feb1cf" containerName="extract"
Jul 14 19:45:21.088726 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.088686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"]
Jul 14 19:45:21.088906 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.088803    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.091655 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.091631    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 14 19:45:21.091900 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.091881    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 14 19:45:21.091955 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.091892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8q6vn\""
Jul 14 19:45:21.241800 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.241760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89w8s\" (UniqueName: \"kubernetes.io/projected/8f7b941d-0668-4e39-82c5-f366368168a8-kube-api-access-89w8s\") pod \"cert-manager-operator-controller-manager-55c444bf86-stltl\" (UID: \"8f7b941d-0668-4e39-82c5-f366368168a8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.241800 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.241804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7b941d-0668-4e39-82c5-f366368168a8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-stltl\" (UID: \"8f7b941d-0668-4e39-82c5-f366368168a8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.342468 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.342367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89w8s\" (UniqueName: \"kubernetes.io/projected/8f7b941d-0668-4e39-82c5-f366368168a8-kube-api-access-89w8s\") pod \"cert-manager-operator-controller-manager-55c444bf86-stltl\" (UID: \"8f7b941d-0668-4e39-82c5-f366368168a8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.342468 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.342427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7b941d-0668-4e39-82c5-f366368168a8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-stltl\" (UID: \"8f7b941d-0668-4e39-82c5-f366368168a8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.342815 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.342796    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f7b941d-0668-4e39-82c5-f366368168a8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-stltl\" (UID: \"8f7b941d-0668-4e39-82c5-f366368168a8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.358580 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.358550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89w8s\" (UniqueName: \"kubernetes.io/projected/8f7b941d-0668-4e39-82c5-f366368168a8-kube-api-access-89w8s\") pod \"cert-manager-operator-controller-manager-55c444bf86-stltl\" (UID: \"8f7b941d-0668-4e39-82c5-f366368168a8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.398385 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.398347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"
Jul 14 19:45:21.578242 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.578205    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl"]
Jul 14 19:45:21.584490 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:45:21.584448    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7b941d_0668_4e39_82c5_f366368168a8.slice/crio-6605fd6a35f167c1e3eb5054a19d85a7988850cbbbbeacd21ec17f73c178004d WatchSource:0}: Error finding container 6605fd6a35f167c1e3eb5054a19d85a7988850cbbbbeacd21ec17f73c178004d: Status 404 returned error can't find the container with id 6605fd6a35f167c1e3eb5054a19d85a7988850cbbbbeacd21ec17f73c178004d
Jul 14 19:45:21.900097 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:21.900059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl" event={"ID":"8f7b941d-0668-4e39-82c5-f366368168a8","Type":"ContainerStarted","Data":"6605fd6a35f167c1e3eb5054a19d85a7988850cbbbbeacd21ec17f73c178004d"}
Jul 14 19:45:23.912205 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:23.912159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl" event={"ID":"8f7b941d-0668-4e39-82c5-f366368168a8","Type":"ContainerStarted","Data":"e3626ed71af14ea68c08a5385fb6f65868f9f51d1a99995cacfdc219d55fe837"}
Jul 14 19:45:23.963664 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:23.963577    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stltl" podStartSLOduration=1.189977508 podStartE2EDuration="2.963554303s" totalImagesPullingTime="1.773576795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:45:23.963096105 +0000 UTC m=+422.274795562" watchObservedRunningTime="2026-07-14 19:45:23.963554303 +0000 UTC m=+422.275253761"
Jul 14 19:45:24.922148 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:24.922116    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"]
Jul 14 19:45:24.924388 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:24.924371    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:24.934433 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:24.934389    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 14 19:45:24.934555 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:24.934434    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5nshw\""
Jul 14 19:45:24.935223 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:24.935208    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 14 19:45:24.960089 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:24.960055    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"]
Jul 14 19:45:25.078462 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.078386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms5bk\" (UniqueName: \"kubernetes.io/projected/4fc8a0e4-f4f7-42cc-b803-436bb203703c-kube-api-access-ms5bk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.078462 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.078471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.078676 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.078522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.179748 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.179650    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms5bk\" (UniqueName: \"kubernetes.io/projected/4fc8a0e4-f4f7-42cc-b803-436bb203703c-kube-api-access-ms5bk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.179748 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.179697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.179987 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.179772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.180137 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.180119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.180137 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.180127    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.214152 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.214114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms5bk\" (UniqueName: \"kubernetes.io/projected/4fc8a0e4-f4f7-42cc-b803-436bb203703c-kube-api-access-ms5bk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.233942 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.233908    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:25.394522 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.394492    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"]
Jul 14 19:45:25.396844 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:45:25.396814    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc8a0e4_f4f7_42cc_b803_436bb203703c.slice/crio-0fedda5bd8cb7c14dfbb3640b884e6167cf8db40df65c450d4ba5bf8f56fce07 WatchSource:0}: Error finding container 0fedda5bd8cb7c14dfbb3640b884e6167cf8db40df65c450d4ba5bf8f56fce07: Status 404 returned error can't find the container with id 0fedda5bd8cb7c14dfbb3640b884e6167cf8db40df65c450d4ba5bf8f56fce07
Jul 14 19:45:25.922997 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.922960    2581 generic.go:356] "Generic (PLEG): container finished" podID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerID="61ff7df0266cc2e85a10b4b4be999a74e2f0aa673af90cab05490cf2cf0f1799" exitCode=0
Jul 14 19:45:25.923441 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.923053    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh" event={"ID":"4fc8a0e4-f4f7-42cc-b803-436bb203703c","Type":"ContainerDied","Data":"61ff7df0266cc2e85a10b4b4be999a74e2f0aa673af90cab05490cf2cf0f1799"}
Jul 14 19:45:25.923441 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:25.923114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh" event={"ID":"4fc8a0e4-f4f7-42cc-b803-436bb203703c","Type":"ContainerStarted","Data":"0fedda5bd8cb7c14dfbb3640b884e6167cf8db40df65c450d4ba5bf8f56fce07"}
Jul 14 19:45:27.937474 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:27.937431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh" event={"ID":"4fc8a0e4-f4f7-42cc-b803-436bb203703c","Type":"ContainerStarted","Data":"a98523021148e3100c632e22540a7471f7063e817a4595e3e8281c16ea4398bb"}
Jul 14 19:45:28.943771 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:28.943739    2581 generic.go:356] "Generic (PLEG): container finished" podID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerID="a98523021148e3100c632e22540a7471f7063e817a4595e3e8281c16ea4398bb" exitCode=0
Jul 14 19:45:28.944176 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:28.943789    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh" event={"ID":"4fc8a0e4-f4f7-42cc-b803-436bb203703c","Type":"ContainerDied","Data":"a98523021148e3100c632e22540a7471f7063e817a4595e3e8281c16ea4398bb"}
Jul 14 19:45:29.950392 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:29.950352    2581 generic.go:356] "Generic (PLEG): container finished" podID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerID="96ab369db8a625a8b6d0d7ec0c3d13ae8ca34f7dc818ad8507fb696ba6e9a6e5" exitCode=0
Jul 14 19:45:29.950828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:29.950435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh" event={"ID":"4fc8a0e4-f4f7-42cc-b803-436bb203703c","Type":"ContainerDied","Data":"96ab369db8a625a8b6d0d7ec0c3d13ae8ca34f7dc818ad8507fb696ba6e9a6e5"}
Jul 14 19:45:31.084823 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.084790    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:31.125235 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.125205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-util\") pod \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") "
Jul 14 19:45:31.125437 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.125267    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-bundle\") pod \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") "
Jul 14 19:45:31.125437 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.125303    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ms5bk\" (UniqueName: \"kubernetes.io/projected/4fc8a0e4-f4f7-42cc-b803-436bb203703c-kube-api-access-ms5bk\") pod \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\" (UID: \"4fc8a0e4-f4f7-42cc-b803-436bb203703c\") "
Jul 14 19:45:31.125788 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.125749    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-bundle" (OuterVolumeSpecName: "bundle") pod "4fc8a0e4-f4f7-42cc-b803-436bb203703c" (UID: "4fc8a0e4-f4f7-42cc-b803-436bb203703c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:45:31.127551 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.127520    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc8a0e4-f4f7-42cc-b803-436bb203703c-kube-api-access-ms5bk" (OuterVolumeSpecName: "kube-api-access-ms5bk") pod "4fc8a0e4-f4f7-42cc-b803-436bb203703c" (UID: "4fc8a0e4-f4f7-42cc-b803-436bb203703c"). InnerVolumeSpecName "kube-api-access-ms5bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:45:31.129856 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.129818    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-util" (OuterVolumeSpecName: "util") pod "4fc8a0e4-f4f7-42cc-b803-436bb203703c" (UID: "4fc8a0e4-f4f7-42cc-b803-436bb203703c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:45:31.226477 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.226378    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-util\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:31.226477 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.226440    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc8a0e4-f4f7-42cc-b803-436bb203703c-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:31.226477 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.226453    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms5bk\" (UniqueName: \"kubernetes.io/projected/4fc8a0e4-f4f7-42cc-b803-436bb203703c-kube-api-access-ms5bk\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:31.483042 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.482947    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-74d86"]
Jul 14 19:45:31.483369 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.483351    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerName="pull"
Jul 14 19:45:31.483487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.483372    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerName="pull"
Jul 14 19:45:31.483487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.483385    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerName="util"
Jul 14 19:45:31.483487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.483392    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerName="util"
Jul 14 19:45:31.483487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.483424    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerName="extract"
Jul 14 19:45:31.483487 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.483432    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerName="extract"
Jul 14 19:45:31.483728 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.483520    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fc8a0e4-f4f7-42cc-b803-436bb203703c" containerName="extract"
Jul 14 19:45:31.485483 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.485460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.489675 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.489651    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 14 19:45:31.489883 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.489864    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-58gp7\""
Jul 14 19:45:31.491273 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.491255    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 14 19:45:31.512884 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.512850    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-74d86"]
Jul 14 19:45:31.528281 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.528246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkhtt\" (UniqueName: \"kubernetes.io/projected/857bb587-8254-46dd-a75a-2f22d4cd42de-kube-api-access-xkhtt\") pod \"cert-manager-webhook-6486ffd997-74d86\" (UID: \"857bb587-8254-46dd-a75a-2f22d4cd42de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.528281 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.528281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/857bb587-8254-46dd-a75a-2f22d4cd42de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-74d86\" (UID: \"857bb587-8254-46dd-a75a-2f22d4cd42de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.629135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.629097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkhtt\" (UniqueName: \"kubernetes.io/projected/857bb587-8254-46dd-a75a-2f22d4cd42de-kube-api-access-xkhtt\") pod \"cert-manager-webhook-6486ffd997-74d86\" (UID: \"857bb587-8254-46dd-a75a-2f22d4cd42de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.629295 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.629141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/857bb587-8254-46dd-a75a-2f22d4cd42de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-74d86\" (UID: \"857bb587-8254-46dd-a75a-2f22d4cd42de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.638179 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.638134    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/857bb587-8254-46dd-a75a-2f22d4cd42de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-74d86\" (UID: \"857bb587-8254-46dd-a75a-2f22d4cd42de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.638322 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.638253    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkhtt\" (UniqueName: \"kubernetes.io/projected/857bb587-8254-46dd-a75a-2f22d4cd42de-kube-api-access-xkhtt\") pod \"cert-manager-webhook-6486ffd997-74d86\" (UID: \"857bb587-8254-46dd-a75a-2f22d4cd42de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.805649 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.805552    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:31.958111 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.958075    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-74d86"]
Jul 14 19:45:31.961780 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:45:31.961753    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod857bb587_8254_46dd_a75a_2f22d4cd42de.slice/crio-b6c8968fbe35fb21b1385ae418ecbf40b8dea8f9572d1ef4af3781990d490390 WatchSource:0}: Error finding container b6c8968fbe35fb21b1385ae418ecbf40b8dea8f9572d1ef4af3781990d490390: Status 404 returned error can't find the container with id b6c8968fbe35fb21b1385ae418ecbf40b8dea8f9572d1ef4af3781990d490390
Jul 14 19:45:31.962742 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.962726    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh"
Jul 14 19:45:31.962815 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.962733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffmwjh" event={"ID":"4fc8a0e4-f4f7-42cc-b803-436bb203703c","Type":"ContainerDied","Data":"0fedda5bd8cb7c14dfbb3640b884e6167cf8db40df65c450d4ba5bf8f56fce07"}
Jul 14 19:45:31.962815 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:31.962786    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fedda5bd8cb7c14dfbb3640b884e6167cf8db40df65c450d4ba5bf8f56fce07"
Jul 14 19:45:32.970522 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:32.970467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-74d86" event={"ID":"857bb587-8254-46dd-a75a-2f22d4cd42de","Type":"ContainerStarted","Data":"b6c8968fbe35fb21b1385ae418ecbf40b8dea8f9572d1ef4af3781990d490390"}
Jul 14 19:45:34.981714 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:34.981678    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-74d86" event={"ID":"857bb587-8254-46dd-a75a-2f22d4cd42de","Type":"ContainerStarted","Data":"5eeb228b6e10ba3955a1d492baed199ebec5557cbf3e9c74df7e8072656bb8ad"}
Jul 14 19:45:34.982377 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:34.981799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:35.685892 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.685827    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-74d86" podStartSLOduration=2.03377577 podStartE2EDuration="4.685806799s" totalImagesPullingTime="2.652031029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:45:35.013999067 +0000 UTC m=+433.325698523" watchObservedRunningTime="2026-07-14 19:45:35.685806799 +0000 UTC m=+433.997506255"
Jul 14 19:45:35.687736 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.687706    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zhz7l"]
Jul 14 19:45:35.689896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.689877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:35.692414 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.692386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7bzxl\""
Jul 14 19:45:35.704374 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.704350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zhz7l"]
Jul 14 19:45:35.765910 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.765876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5-bound-sa-token\") pod \"cert-manager-55b4d954d5-zhz7l\" (UID: \"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5\") " pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:35.766073 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.765934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8776\" (UniqueName: \"kubernetes.io/projected/6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5-kube-api-access-n8776\") pod \"cert-manager-55b4d954d5-zhz7l\" (UID: \"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5\") " pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:35.867065 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.867018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5-bound-sa-token\") pod \"cert-manager-55b4d954d5-zhz7l\" (UID: \"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5\") " pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:35.867265 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.867081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8776\" (UniqueName: \"kubernetes.io/projected/6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5-kube-api-access-n8776\") pod \"cert-manager-55b4d954d5-zhz7l\" (UID: \"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5\") " pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:35.883871 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.883842    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8776\" (UniqueName: \"kubernetes.io/projected/6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5-kube-api-access-n8776\") pod \"cert-manager-55b4d954d5-zhz7l\" (UID: \"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5\") " pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:35.893742 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.893709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5-bound-sa-token\") pod \"cert-manager-55b4d954d5-zhz7l\" (UID: \"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5\") " pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:35.998668 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:35.998618    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zhz7l"
Jul 14 19:45:36.148460 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:36.148426    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zhz7l"]
Jul 14 19:45:36.151646 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:45:36.151607    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d4bfc07_14ab_48bf_a4c9_6a4afa4c36c5.slice/crio-dd0280e2464250566d43a6961b5aa42b6c536543c7b50e5a459be7a097050c0f WatchSource:0}: Error finding container dd0280e2464250566d43a6961b5aa42b6c536543c7b50e5a459be7a097050c0f: Status 404 returned error can't find the container with id dd0280e2464250566d43a6961b5aa42b6c536543c7b50e5a459be7a097050c0f
Jul 14 19:45:36.998330 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:36.998289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zhz7l" event={"ID":"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5","Type":"ContainerStarted","Data":"fcb65b5317ef0d7c6996aa2543e727e781165a83fdc93867af86db5aaee50a6e"}
Jul 14 19:45:36.998330 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:36.998338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zhz7l" event={"ID":"6d4bfc07-14ab-48bf-a4c9-6a4afa4c36c5","Type":"ContainerStarted","Data":"dd0280e2464250566d43a6961b5aa42b6c536543c7b50e5a459be7a097050c0f"}
Jul 14 19:45:37.033542 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:37.033471    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-zhz7l" podStartSLOduration=2.033450102 podStartE2EDuration="2.033450102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:45:37.030561595 +0000 UTC m=+435.342261051" watchObservedRunningTime="2026-07-14 19:45:37.033450102 +0000 UTC m=+435.345149559"
Jul 14 19:45:40.988971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:40.988941    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-74d86"
Jul 14 19:45:44.655530 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.655440    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"]
Jul 14 19:45:44.658070 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.658045    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.660629 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.660603    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 14 19:45:44.660775 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.660671    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 14 19:45:44.661305 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.661290    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5nshw\""
Jul 14 19:45:44.670144 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.670115    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"]
Jul 14 19:45:44.748713 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.748672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.748902 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.748723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.748902 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.748813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrl2\" (UniqueName: \"kubernetes.io/projected/5781ccf6-f954-41ab-9051-bed15d26ec56-kube-api-access-ncrl2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.850017 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.849974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.850196 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.850034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrl2\" (UniqueName: \"kubernetes.io/projected/5781ccf6-f954-41ab-9051-bed15d26ec56-kube-api-access-ncrl2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.850196 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.850112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.850391 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.850370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.850521 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.850498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.860383 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.860354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrl2\" (UniqueName: \"kubernetes.io/projected/5781ccf6-f954-41ab-9051-bed15d26ec56-kube-api-access-ncrl2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:44.967976 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:44.967932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:45.117177 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:45.117149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"]
Jul 14 19:45:45.119357 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:45:45.119318    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5781ccf6_f954_41ab_9051_bed15d26ec56.slice/crio-3f1bacfb0753f06efb5dbf342ab5daf68e0a2a465fb88a3ea2fad6b28a28d91b WatchSource:0}: Error finding container 3f1bacfb0753f06efb5dbf342ab5daf68e0a2a465fb88a3ea2fad6b28a28d91b: Status 404 returned error can't find the container with id 3f1bacfb0753f06efb5dbf342ab5daf68e0a2a465fb88a3ea2fad6b28a28d91b
Jul 14 19:45:46.043787 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:46.043752    2581 generic.go:356] "Generic (PLEG): container finished" podID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerID="80fa3f96508eed03e9f10a13b4aa3fd1a217497ac1278b4a64e6b692bf92d56a" exitCode=0
Jul 14 19:45:46.044221 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:46.043810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb" event={"ID":"5781ccf6-f954-41ab-9051-bed15d26ec56","Type":"ContainerDied","Data":"80fa3f96508eed03e9f10a13b4aa3fd1a217497ac1278b4a64e6b692bf92d56a"}
Jul 14 19:45:46.044221 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:46.043846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb" event={"ID":"5781ccf6-f954-41ab-9051-bed15d26ec56","Type":"ContainerStarted","Data":"3f1bacfb0753f06efb5dbf342ab5daf68e0a2a465fb88a3ea2fad6b28a28d91b"}
Jul 14 19:45:47.050080 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:47.049973    2581 generic.go:356] "Generic (PLEG): container finished" podID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerID="5ea1d72aaa69656d3947c416cafb100c3209e2afa8cca5eb0ea6a8ec1a08743e" exitCode=0
Jul 14 19:45:47.050080 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:47.050062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb" event={"ID":"5781ccf6-f954-41ab-9051-bed15d26ec56","Type":"ContainerDied","Data":"5ea1d72aaa69656d3947c416cafb100c3209e2afa8cca5eb0ea6a8ec1a08743e"}
Jul 14 19:45:48.057284 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:48.057250    2581 generic.go:356] "Generic (PLEG): container finished" podID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerID="48db33add908e4f190c39549858d1f6b5ed733f5e3f912d199d100da7f8bfecd" exitCode=0
Jul 14 19:45:48.057716 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:48.057327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb" event={"ID":"5781ccf6-f954-41ab-9051-bed15d26ec56","Type":"ContainerDied","Data":"48db33add908e4f190c39549858d1f6b5ed733f5e3f912d199d100da7f8bfecd"}
Jul 14 19:45:49.203859 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.203828    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:45:49.287331 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.287286    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-bundle\") pod \"5781ccf6-f954-41ab-9051-bed15d26ec56\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") "
Jul 14 19:45:49.287531 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.287374    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncrl2\" (UniqueName: \"kubernetes.io/projected/5781ccf6-f954-41ab-9051-bed15d26ec56-kube-api-access-ncrl2\") pod \"5781ccf6-f954-41ab-9051-bed15d26ec56\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") "
Jul 14 19:45:49.287531 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.287438    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-util\") pod \"5781ccf6-f954-41ab-9051-bed15d26ec56\" (UID: \"5781ccf6-f954-41ab-9051-bed15d26ec56\") "
Jul 14 19:45:49.288083 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.288050    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-bundle" (OuterVolumeSpecName: "bundle") pod "5781ccf6-f954-41ab-9051-bed15d26ec56" (UID: "5781ccf6-f954-41ab-9051-bed15d26ec56"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:45:49.289614 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.289591    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5781ccf6-f954-41ab-9051-bed15d26ec56-kube-api-access-ncrl2" (OuterVolumeSpecName: "kube-api-access-ncrl2") pod "5781ccf6-f954-41ab-9051-bed15d26ec56" (UID: "5781ccf6-f954-41ab-9051-bed15d26ec56"). InnerVolumeSpecName "kube-api-access-ncrl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:45:49.295654 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.295625    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-util" (OuterVolumeSpecName: "util") pod "5781ccf6-f954-41ab-9051-bed15d26ec56" (UID: "5781ccf6-f954-41ab-9051-bed15d26ec56"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:45:49.388342 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.388305    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:49.388342 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.388336    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncrl2\" (UniqueName: \"kubernetes.io/projected/5781ccf6-f954-41ab-9051-bed15d26ec56-kube-api-access-ncrl2\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:49.388342 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:49.388346    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5781ccf6-f954-41ab-9051-bed15d26ec56-util\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:45:50.068591 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:50.068554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb" event={"ID":"5781ccf6-f954-41ab-9051-bed15d26ec56","Type":"ContainerDied","Data":"3f1bacfb0753f06efb5dbf342ab5daf68e0a2a465fb88a3ea2fad6b28a28d91b"}
Jul 14 19:45:50.068781 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:50.068605    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f1bacfb0753f06efb5dbf342ab5daf68e0a2a465fb88a3ea2fad6b28a28d91b"
Jul 14 19:45:50.068781 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:45:50.068605    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5bxlb"
Jul 14 19:46:01.232459 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232414    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"]
Jul 14 19:46:01.232956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232839    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerName="util"
Jul 14 19:46:01.232956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232853    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerName="util"
Jul 14 19:46:01.232956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232863    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerName="pull"
Jul 14 19:46:01.232956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232869    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerName="pull"
Jul 14 19:46:01.232956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232874    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerName="extract"
Jul 14 19:46:01.232956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232880    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerName="extract"
Jul 14 19:46:01.232956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.232949    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5781ccf6-f954-41ab-9051-bed15d26ec56" containerName="extract"
Jul 14 19:46:01.237722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.237695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.240874 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.240849    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 19:46:01.241023 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.240855    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 19:46:01.241156 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.241140    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 19:46:01.241230 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.241202    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wl8d8\""
Jul 14 19:46:01.245554 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.245535    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 19:46:01.261601 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.261567    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"]
Jul 14 19:46:01.398743 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.398702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60bb432d-0467-4f8b-9fc7-dec8390bf954-tmp\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.398743 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.398755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60bb432d-0467-4f8b-9fc7-dec8390bf954-manifests\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.398975 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.398865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc299\" (UniqueName: \"kubernetes.io/projected/60bb432d-0467-4f8b-9fc7-dec8390bf954-kube-api-access-dc299\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.398975 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.398950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60bb432d-0467-4f8b-9fc7-dec8390bf954-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.399064 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.399001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60bb432d-0467-4f8b-9fc7-dec8390bf954-webhook-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.499729 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.499631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc299\" (UniqueName: \"kubernetes.io/projected/60bb432d-0467-4f8b-9fc7-dec8390bf954-kube-api-access-dc299\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.499729 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.499695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60bb432d-0467-4f8b-9fc7-dec8390bf954-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.499928 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.499872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60bb432d-0467-4f8b-9fc7-dec8390bf954-webhook-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.499989 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.499926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60bb432d-0467-4f8b-9fc7-dec8390bf954-tmp\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.500107 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.500080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60bb432d-0467-4f8b-9fc7-dec8390bf954-manifests\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.500423 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.500375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60bb432d-0467-4f8b-9fc7-dec8390bf954-manifests\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.502089 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.502061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60bb432d-0467-4f8b-9fc7-dec8390bf954-tmp\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.502457 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.502432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60bb432d-0467-4f8b-9fc7-dec8390bf954-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.502457 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.502444    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60bb432d-0467-4f8b-9fc7-dec8390bf954-webhook-cert\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.514653 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.514632    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc299\" (UniqueName: \"kubernetes.io/projected/60bb432d-0467-4f8b-9fc7-dec8390bf954-kube-api-access-dc299\") pod \"opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf\" (UID: \"60bb432d-0467-4f8b-9fc7-dec8390bf954\") " pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.548752 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.548705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:01.702187 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:01.702145    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"]
Jul 14 19:46:01.705794 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:46:01.705764    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bb432d_0467_4f8b_9fc7_dec8390bf954.slice/crio-a8f053c7616f4364902c4a935700e9dd709f2d04d2aa5f5d20b57700fcc7074e WatchSource:0}: Error finding container a8f053c7616f4364902c4a935700e9dd709f2d04d2aa5f5d20b57700fcc7074e: Status 404 returned error can't find the container with id a8f053c7616f4364902c4a935700e9dd709f2d04d2aa5f5d20b57700fcc7074e
Jul 14 19:46:02.122581 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:02.122543    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf" event={"ID":"60bb432d-0467-4f8b-9fc7-dec8390bf954","Type":"ContainerStarted","Data":"a8f053c7616f4364902c4a935700e9dd709f2d04d2aa5f5d20b57700fcc7074e"}
Jul 14 19:46:05.146084 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:05.145977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf" event={"ID":"60bb432d-0467-4f8b-9fc7-dec8390bf954","Type":"ContainerStarted","Data":"020ec449c8701073c680a20fc0777d630b77d00d066a89f85ca76d4d36c37bea"}
Jul 14 19:46:06.152654 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:06.152620    2581 generic.go:356] "Generic (PLEG): container finished" podID="60bb432d-0467-4f8b-9fc7-dec8390bf954" containerID="020ec449c8701073c680a20fc0777d630b77d00d066a89f85ca76d4d36c37bea" exitCode=0
Jul 14 19:46:06.153038 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:06.152675    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf" event={"ID":"60bb432d-0467-4f8b-9fc7-dec8390bf954","Type":"ContainerDied","Data":"020ec449c8701073c680a20fc0777d630b77d00d066a89f85ca76d4d36c37bea"}
Jul 14 19:46:08.164548 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:08.164510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf" event={"ID":"60bb432d-0467-4f8b-9fc7-dec8390bf954","Type":"ContainerStarted","Data":"6e72ad56989710520199611e3849c5dae2778113bb75fe60684367c48d4f588c"}
Jul 14 19:46:08.165068 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:08.164626    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:19.172530 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:19.172498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf"
Jul 14 19:46:19.204858 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:19.204798    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf" podStartSLOduration=12.874478866 podStartE2EDuration="18.204780099s" totalImagesPullingTime="4.261248149s" totalInitContainerRuntime="1.069053084s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:08.200740087 +0000 UTC m=+466.512439543" watchObservedRunningTime="2026-07-14 19:46:19.204780099 +0000 UTC m=+477.516479554"
Jul 14 19:46:22.491486 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.491450    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"]
Jul 14 19:46:22.493799 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.493780    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.498200 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.498177    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 19:46:22.498580 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.498561    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5zmfb\""
Jul 14 19:46:22.498997 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.498979    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 19:46:22.499626 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.499601    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 19:46:22.504139 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.504116    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 19:46:22.522293 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.522265    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"]
Jul 14 19:46:22.587565 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.587521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd98c15a-ae0e-4c56-82e2-482bafa874a3-tmp\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.587765 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.587593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd98c15a-ae0e-4c56-82e2-482bafa874a3-tls-certs\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.587765 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.587643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j5lt\" (UniqueName: \"kubernetes.io/projected/dd98c15a-ae0e-4c56-82e2-482bafa874a3-kube-api-access-5j5lt\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.688781 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.688742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd98c15a-ae0e-4c56-82e2-482bafa874a3-tmp\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.688985 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.688803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd98c15a-ae0e-4c56-82e2-482bafa874a3-tls-certs\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.688985 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.688849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j5lt\" (UniqueName: \"kubernetes.io/projected/dd98c15a-ae0e-4c56-82e2-482bafa874a3-kube-api-access-5j5lt\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.691099 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.691075    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd98c15a-ae0e-4c56-82e2-482bafa874a3-tmp\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.691360 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.691341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd98c15a-ae0e-4c56-82e2-482bafa874a3-tls-certs\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.732068 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.732034    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j5lt\" (UniqueName: \"kubernetes.io/projected/dd98c15a-ae0e-4c56-82e2-482bafa874a3-kube-api-access-5j5lt\") pod \"kube-auth-proxy-59f575c9d9-w89dt\" (UID: \"dd98c15a-ae0e-4c56-82e2-482bafa874a3\") " pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.803768 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.803669    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"
Jul 14 19:46:22.993020 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:46:22.992977    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd98c15a_ae0e_4c56_82e2_482bafa874a3.slice/crio-c5c58b4613bc17bf59b203005a4396848b7b93854ea006f9b213dce253dcc876 WatchSource:0}: Error finding container c5c58b4613bc17bf59b203005a4396848b7b93854ea006f9b213dce253dcc876: Status 404 returned error can't find the container with id c5c58b4613bc17bf59b203005a4396848b7b93854ea006f9b213dce253dcc876
Jul 14 19:46:22.994130 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.994105    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-njgf6"]
Jul 14 19:46:22.997946 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.997345    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt"]
Jul 14 19:46:22.997946 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:22.997488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.006216 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.006186    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 14 19:46:23.006562 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.006538    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-g8gsk\""
Jul 14 19:46:23.040805 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.040774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-njgf6"]
Jul 14 19:46:23.091956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.091862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gccfs\" (UniqueName: \"kubernetes.io/projected/08542d4a-edbd-4d83-8bf3-bcec89cc3876-kube-api-access-gccfs\") pod \"odh-model-controller-7998c67dc4-njgf6\" (UID: \"08542d4a-edbd-4d83-8bf3-bcec89cc3876\") " pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.092124 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.091970    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08542d4a-edbd-4d83-8bf3-bcec89cc3876-cert\") pod \"odh-model-controller-7998c67dc4-njgf6\" (UID: \"08542d4a-edbd-4d83-8bf3-bcec89cc3876\") " pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.193238 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.193200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08542d4a-edbd-4d83-8bf3-bcec89cc3876-cert\") pod \"odh-model-controller-7998c67dc4-njgf6\" (UID: \"08542d4a-edbd-4d83-8bf3-bcec89cc3876\") " pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.193448 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.193247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gccfs\" (UniqueName: \"kubernetes.io/projected/08542d4a-edbd-4d83-8bf3-bcec89cc3876-kube-api-access-gccfs\") pod \"odh-model-controller-7998c67dc4-njgf6\" (UID: \"08542d4a-edbd-4d83-8bf3-bcec89cc3876\") " pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.210786 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.210750    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 14 19:46:23.216319 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.216287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08542d4a-edbd-4d83-8bf3-bcec89cc3876-cert\") pod \"odh-model-controller-7998c67dc4-njgf6\" (UID: \"08542d4a-edbd-4d83-8bf3-bcec89cc3876\") " pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.226302 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.226270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gccfs\" (UniqueName: \"kubernetes.io/projected/08542d4a-edbd-4d83-8bf3-bcec89cc3876-kube-api-access-gccfs\") pod \"odh-model-controller-7998c67dc4-njgf6\" (UID: \"08542d4a-edbd-4d83-8bf3-bcec89cc3876\") " pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.236505 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.236471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt" event={"ID":"dd98c15a-ae0e-4c56-82e2-482bafa874a3","Type":"ContainerStarted","Data":"c5c58b4613bc17bf59b203005a4396848b7b93854ea006f9b213dce253dcc876"}
Jul 14 19:46:23.314913 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.314883    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-g8gsk\""
Jul 14 19:46:23.322892 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.322864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:23.502431 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:23.502379    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-njgf6"]
Jul 14 19:46:23.504847 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:46:23.504816    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08542d4a_edbd_4d83_8bf3_bcec89cc3876.slice/crio-ccb921bb29a782f89d809a9e7fcb466dacd2a7ce272df33cc923d89c2037ac31 WatchSource:0}: Error finding container ccb921bb29a782f89d809a9e7fcb466dacd2a7ce272df33cc923d89c2037ac31: Status 404 returned error can't find the container with id ccb921bb29a782f89d809a9e7fcb466dacd2a7ce272df33cc923d89c2037ac31
Jul 14 19:46:24.243158 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:24.243119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-njgf6" event={"ID":"08542d4a-edbd-4d83-8bf3-bcec89cc3876","Type":"ContainerStarted","Data":"ccb921bb29a782f89d809a9e7fcb466dacd2a7ce272df33cc923d89c2037ac31"}
Jul 14 19:46:25.722951 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.722908    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"]
Jul 14 19:46:25.733210 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.733172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.736121 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.736081    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 14 19:46:25.736288 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.736128    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 14 19:46:25.736483 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.736458    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 14 19:46:25.736601 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.736470    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n627z\""
Jul 14 19:46:25.736672 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.736615    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 14 19:46:25.748692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.748658    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"]
Jul 14 19:46:25.817513 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.817415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtsfk\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-kube-api-access-xtsfk\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.817513 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.817488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecebed5f-1f64-4249-aea9-d59934063afb-local-certs\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.817745 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.817615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-istio-token\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.817745 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.817691    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.817824 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.817745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-cacerts\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.817824 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.817803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.817929 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.817854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.918555 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.918514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.918768 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.918619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtsfk\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-kube-api-access-xtsfk\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.918768 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.918658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecebed5f-1f64-4249-aea9-d59934063afb-local-certs\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.918768 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.918712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-istio-token\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.918768 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.918748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.919017 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.918789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-cacerts\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.919017 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.918823    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.919676 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.919612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.921495 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.921447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecebed5f-1f64-4249-aea9-d59934063afb-local-certs\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.921687 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.921662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.921846 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.921820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-cacerts\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.922176 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.922154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.932030 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.932002    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-istio-token\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:25.933642 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:25.933612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtsfk\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-kube-api-access-xtsfk\") pod \"istiod-openshift-gateway-7574459656-5p2jt\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:26.046293 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:26.046202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:27.313987 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:27.313921    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 19:46:27.479698 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:27.479601    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"]
Jul 14 19:46:27.486743 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:46:27.486692    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecebed5f_1f64_4249_aea9_d59934063afb.slice/crio-b907953f04905a68f7e7942cfe95c83d56a55bf5ea91f7977d34ccdee7a1f9c3 WatchSource:0}: Error finding container b907953f04905a68f7e7942cfe95c83d56a55bf5ea91f7977d34ccdee7a1f9c3: Status 404 returned error can't find the container with id b907953f04905a68f7e7942cfe95c83d56a55bf5ea91f7977d34ccdee7a1f9c3
Jul 14 19:46:28.269353 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:28.269099    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-njgf6" event={"ID":"08542d4a-edbd-4d83-8bf3-bcec89cc3876","Type":"ContainerStarted","Data":"c8281eda3414b3698a6c30dcdc3f420a5bf98a5d099978f430b5c447579fb6fb"}
Jul 14 19:46:28.269353 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:28.269205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:28.271595 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:28.271559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt" event={"ID":"dd98c15a-ae0e-4c56-82e2-482bafa874a3","Type":"ContainerStarted","Data":"213c1a0ef4eb8f4a310cca5388c7121ac5eba7f6f0ffa683de301bec5f775f02"}
Jul 14 19:46:28.273184 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:28.273154    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt" event={"ID":"ecebed5f-1f64-4249-aea9-d59934063afb","Type":"ContainerStarted","Data":"b907953f04905a68f7e7942cfe95c83d56a55bf5ea91f7977d34ccdee7a1f9c3"}
Jul 14 19:46:28.297683 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:28.297606    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-njgf6" podStartSLOduration=2.497852152 podStartE2EDuration="6.297583206s" totalImagesPullingTime="3.799731054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:28.295697552 +0000 UTC m=+486.607397008" watchObservedRunningTime="2026-07-14 19:46:28.297583206 +0000 UTC m=+486.609282662"
Jul 14 19:46:28.331690 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:28.331613    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59f575c9d9-w89dt" podStartSLOduration=2.026990036 podStartE2EDuration="6.331586961s" totalImagesPullingTime="4.304596925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:28.331249764 +0000 UTC m=+486.642949213" watchObservedRunningTime="2026-07-14 19:46:28.331586961 +0000 UTC m=+486.643286420"
Jul 14 19:46:29.976934 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:29.976898    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:46:29.977202 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:29.976969    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:46:30.286395 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:30.286302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt" event={"ID":"ecebed5f-1f64-4249-aea9-d59934063afb","Type":"ContainerStarted","Data":"e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6"}
Jul 14 19:46:30.286571 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:30.286432    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:31.292727 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:31.292689    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5p2jt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 14 19:46:31.293127 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:31.292746    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt" podUID="ecebed5f-1f64-4249-aea9-d59934063afb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 19:46:33.175747 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.175686    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt" podStartSLOduration=5.688104994 podStartE2EDuration="8.175667252s" totalImagesPullingTime="2.487562258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:30.316880343 +0000 UTC m=+488.628579803" watchObservedRunningTime="2026-07-14 19:46:33.175667252 +0000 UTC m=+491.487366708"
Jul 14 19:46:33.176121 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.176088    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"]
Jul 14 19:46:33.179234 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.179215    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.214030 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.213992    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-sj8sw\""
Jul 14 19:46:33.235528 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.235492    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"]
Jul 14 19:46:33.289453 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289638 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bs7d\" (UniqueName: \"kubernetes.io/projected/28fd6f37-f91f-4899-b011-da9c60ca0875-kube-api-access-5bs7d\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289638 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289638 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289741 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289741 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289741 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289741 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fd6f37-f91f-4899-b011-da9c60ca0875-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.289741 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.289737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390575 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390773 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390773 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390773 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fd6f37-f91f-4899-b011-da9c60ca0875-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390773 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390991 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390991 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390859    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bs7d\" (UniqueName: \"kubernetes.io/projected/28fd6f37-f91f-4899-b011-da9c60ca0875-kube-api-access-5bs7d\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390991 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.390991 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.390962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.391282 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.391257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.391355 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.391303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.391432 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.391375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.391432 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.391390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.391790 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.391772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fd6f37-f91f-4899-b011-da9c60ca0875-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.392989 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.392967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.393311 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.393292    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.470819 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.470787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bs7d\" (UniqueName: \"kubernetes.io/projected/28fd6f37-f91f-4899-b011-da9c60ca0875-kube-api-access-5bs7d\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.473375 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.473343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fd6f37-f91f-4899-b011-da9c60ca0875-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45jd7s\" (UID: \"28fd6f37-f91f-4899-b011-da9c60ca0875\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.490777 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.490347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:33.701160 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:33.701117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"]
Jul 14 19:46:33.704903 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:46:33.704870    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fd6f37_f91f_4899_b011_da9c60ca0875.slice/crio-41efdeeb24f7e36b58a6b1f7081445b74e02ccfab3bcf108487d1d3928be2efa WatchSource:0}: Error finding container 41efdeeb24f7e36b58a6b1f7081445b74e02ccfab3bcf108487d1d3928be2efa: Status 404 returned error can't find the container with id 41efdeeb24f7e36b58a6b1f7081445b74e02ccfab3bcf108487d1d3928be2efa
Jul 14 19:46:34.292907 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:34.292875    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:34.311834 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:34.311798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s" event={"ID":"28fd6f37-f91f-4899-b011-da9c60ca0875","Type":"ContainerStarted","Data":"41efdeeb24f7e36b58a6b1f7081445b74e02ccfab3bcf108487d1d3928be2efa"}
Jul 14 19:46:35.477502 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.477466    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"]
Jul 14 19:46:35.484844 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.484813    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:35.512012 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.511844    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2kmtw\""
Jul 14 19:46:35.513168 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.513138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vvr8\" (UniqueName: \"kubernetes.io/projected/c5ee1074-bafe-4acf-bd7f-4235b8be00f0-kube-api-access-8vvr8\") pod \"kserve-controller-manager-d6f4b65b8-5jdhc\" (UID: \"c5ee1074-bafe-4acf-bd7f-4235b8be00f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:35.513323 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.513294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5ee1074-bafe-4acf-bd7f-4235b8be00f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-5jdhc\" (UID: \"c5ee1074-bafe-4acf-bd7f-4235b8be00f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:35.547609 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.547570    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"]
Jul 14 19:46:35.550705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.550671    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 14 19:46:35.551815 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.551794    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"]
Jul 14 19:46:35.552321 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.552306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:35.557840 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.557816    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 14 19:46:35.574787 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.574756    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t6nrr\""
Jul 14 19:46:35.613233 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.613194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"]
Jul 14 19:46:35.614374 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.614341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5ee1074-bafe-4acf-bd7f-4235b8be00f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-5jdhc\" (UID: \"c5ee1074-bafe-4acf-bd7f-4235b8be00f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:35.614550 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.614429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489ece5a-4af8-4670-bb4a-76119587279d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bwjw\" (UID: \"489ece5a-4af8-4670-bb4a-76119587279d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:35.614550 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.614478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vvr8\" (UniqueName: \"kubernetes.io/projected/c5ee1074-bafe-4acf-bd7f-4235b8be00f0-kube-api-access-8vvr8\") pod \"kserve-controller-manager-d6f4b65b8-5jdhc\" (UID: \"c5ee1074-bafe-4acf-bd7f-4235b8be00f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:35.614659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.614575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmlbq\" (UniqueName: \"kubernetes.io/projected/489ece5a-4af8-4670-bb4a-76119587279d-kube-api-access-pmlbq\") pod \"llmisvc-controller-manager-777c6bd7d5-4bwjw\" (UID: \"489ece5a-4af8-4670-bb4a-76119587279d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:35.617310 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.617283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5ee1074-bafe-4acf-bd7f-4235b8be00f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-5jdhc\" (UID: \"c5ee1074-bafe-4acf-bd7f-4235b8be00f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:35.659200 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.659167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vvr8\" (UniqueName: \"kubernetes.io/projected/c5ee1074-bafe-4acf-bd7f-4235b8be00f0-kube-api-access-8vvr8\") pod \"kserve-controller-manager-d6f4b65b8-5jdhc\" (UID: \"c5ee1074-bafe-4acf-bd7f-4235b8be00f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:35.715280 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.715241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489ece5a-4af8-4670-bb4a-76119587279d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bwjw\" (UID: \"489ece5a-4af8-4670-bb4a-76119587279d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:35.715500 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.715346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmlbq\" (UniqueName: \"kubernetes.io/projected/489ece5a-4af8-4670-bb4a-76119587279d-kube-api-access-pmlbq\") pod \"llmisvc-controller-manager-777c6bd7d5-4bwjw\" (UID: \"489ece5a-4af8-4670-bb4a-76119587279d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:35.715500 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:46:35.715440    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 14 19:46:35.715642 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:46:35.715533    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/489ece5a-4af8-4670-bb4a-76119587279d-cert podName:489ece5a-4af8-4670-bb4a-76119587279d nodeName:}" failed. No retries permitted until 2026-07-14 19:46:36.215510675 +0000 UTC m=+494.527210127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/489ece5a-4af8-4670-bb4a-76119587279d-cert") pod "llmisvc-controller-manager-777c6bd7d5-4bwjw" (UID: "489ece5a-4af8-4670-bb4a-76119587279d") : secret "llmisvc-webhook-server-cert" not found
Jul 14 19:46:35.738457 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.738349    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmlbq\" (UniqueName: \"kubernetes.io/projected/489ece5a-4af8-4670-bb4a-76119587279d-kube-api-access-pmlbq\") pod \"llmisvc-controller-manager-777c6bd7d5-4bwjw\" (UID: \"489ece5a-4af8-4670-bb4a-76119587279d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:35.797201 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:35.797158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:36.219374 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.219330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489ece5a-4af8-4670-bb4a-76119587279d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bwjw\" (UID: \"489ece5a-4af8-4670-bb4a-76119587279d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:36.222109 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.222073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489ece5a-4af8-4670-bb4a-76119587279d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bwjw\" (UID: \"489ece5a-4af8-4670-bb4a-76119587279d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:36.417940 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.417880    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:46:36.418080 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.417990    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:46:36.418080 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.418032    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:46:36.464167 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.464087    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:36.531332 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.530142    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"]
Jul 14 19:46:36.532283 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:46:36.532253    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ee1074_bafe_4acf_bd7f_4235b8be00f0.slice/crio-0bf7113b250e0474bd2a8dfd9e7dbcdeadc2b516a6789b3cd8b2310eca39b820 WatchSource:0}: Error finding container 0bf7113b250e0474bd2a8dfd9e7dbcdeadc2b516a6789b3cd8b2310eca39b820: Status 404 returned error can't find the container with id 0bf7113b250e0474bd2a8dfd9e7dbcdeadc2b516a6789b3cd8b2310eca39b820
Jul 14 19:46:36.733353 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:36.733324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"]
Jul 14 19:46:36.735437 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:46:36.735379    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod489ece5a_4af8_4670_bb4a_76119587279d.slice/crio-0f1781a55d78b11337719b6b5c0f940b0ef380e94a9fbba0f59f877e28682f45 WatchSource:0}: Error finding container 0f1781a55d78b11337719b6b5c0f940b0ef380e94a9fbba0f59f877e28682f45: Status 404 returned error can't find the container with id 0f1781a55d78b11337719b6b5c0f940b0ef380e94a9fbba0f59f877e28682f45
Jul 14 19:46:37.329943 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:37.329894    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw" event={"ID":"489ece5a-4af8-4670-bb4a-76119587279d","Type":"ContainerStarted","Data":"0f1781a55d78b11337719b6b5c0f940b0ef380e94a9fbba0f59f877e28682f45"}
Jul 14 19:46:37.331435 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:37.331393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s" event={"ID":"28fd6f37-f91f-4899-b011-da9c60ca0875","Type":"ContainerStarted","Data":"d8ab7fafcfe2e3f9e7362311a77dd044615e7d5b8c772be3095ed3cdec971fd0"}
Jul 14 19:46:37.337367 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:37.337333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc" event={"ID":"c5ee1074-bafe-4acf-bd7f-4235b8be00f0","Type":"ContainerStarted","Data":"0bf7113b250e0474bd2a8dfd9e7dbcdeadc2b516a6789b3cd8b2310eca39b820"}
Jul 14 19:46:37.396705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:37.396655    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s" podStartSLOduration=1.686334072 podStartE2EDuration="4.396636575s" totalImagesPullingTime="2.710302503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:37.395052576 +0000 UTC m=+495.706752033" watchObservedRunningTime="2026-07-14 19:46:37.396636575 +0000 UTC m=+495.708336030"
Jul 14 19:46:37.490938 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:37.490894    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:37.497215 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:37.497177    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:38.338849 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:38.338812    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:38.340103 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:38.340079    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45jd7s"
Jul 14 19:46:39.281359 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:39.281324    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-njgf6"
Jul 14 19:46:40.354145 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.354046    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc" event={"ID":"c5ee1074-bafe-4acf-bd7f-4235b8be00f0","Type":"ContainerStarted","Data":"230d76c3724a448df96a4b425817435b1ad2caa4c663459767fdbb91c2f396b0"}
Jul 14 19:46:40.354670 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.354315    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:46:40.374194 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.374149    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"]
Jul 14 19:46:40.374708 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.374663    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt" podUID="ecebed5f-1f64-4249-aea9-d59934063afb" containerName="discovery" containerID="cri-o://e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6" gracePeriod=30
Jul 14 19:46:40.441192 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.441137    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc" podStartSLOduration=2.566572388 podStartE2EDuration="5.441116912s" totalImagesPullingTime="2.874544524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:40.439309575 +0000 UTC m=+498.751009031" watchObservedRunningTime="2026-07-14 19:46:40.441116912 +0000 UTC m=+498.752816368"
Jul 14 19:46:40.644143 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.644113    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:40.765637 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.765604    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-istio-token\") pod \"ecebed5f-1f64-4249-aea9-d59934063afb\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") "
Jul 14 19:46:40.765828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.765652    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecebed5f-1f64-4249-aea9-d59934063afb-local-certs\") pod \"ecebed5f-1f64-4249-aea9-d59934063afb\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") "
Jul 14 19:46:40.765828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.765705    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-ca-configmap\") pod \"ecebed5f-1f64-4249-aea9-d59934063afb\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") "
Jul 14 19:46:40.765828 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.765817    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtsfk\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-kube-api-access-xtsfk\") pod \"ecebed5f-1f64-4249-aea9-d59934063afb\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") "
Jul 14 19:46:40.765976 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.765889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-dns-cert\") pod \"ecebed5f-1f64-4249-aea9-d59934063afb\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") "
Jul 14 19:46:40.765976 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.765945    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-cacerts\") pod \"ecebed5f-1f64-4249-aea9-d59934063afb\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") "
Jul 14 19:46:40.765976 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.765968    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-kubeconfig\") pod \"ecebed5f-1f64-4249-aea9-d59934063afb\" (UID: \"ecebed5f-1f64-4249-aea9-d59934063afb\") "
Jul 14 19:46:40.766124 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.766104    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ecebed5f-1f64-4249-aea9-d59934063afb" (UID: "ecebed5f-1f64-4249-aea9-d59934063afb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:46:40.766353 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.766318    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-ca-configmap\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:46:40.768799 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.768754    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecebed5f-1f64-4249-aea9-d59934063afb-local-certs" (OuterVolumeSpecName: "local-certs") pod "ecebed5f-1f64-4249-aea9-d59934063afb" (UID: "ecebed5f-1f64-4249-aea9-d59934063afb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:46:40.768799 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.768770    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-kube-api-access-xtsfk" (OuterVolumeSpecName: "kube-api-access-xtsfk") pod "ecebed5f-1f64-4249-aea9-d59934063afb" (UID: "ecebed5f-1f64-4249-aea9-d59934063afb"). InnerVolumeSpecName "kube-api-access-xtsfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:46:40.768799 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.768785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-istio-token" (OuterVolumeSpecName: "istio-token") pod "ecebed5f-1f64-4249-aea9-d59934063afb" (UID: "ecebed5f-1f64-4249-aea9-d59934063afb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:46:40.769010 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.768795    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-cacerts" (OuterVolumeSpecName: "cacerts") pod "ecebed5f-1f64-4249-aea9-d59934063afb" (UID: "ecebed5f-1f64-4249-aea9-d59934063afb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:46:40.769010 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.768996    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ecebed5f-1f64-4249-aea9-d59934063afb" (UID: "ecebed5f-1f64-4249-aea9-d59934063afb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:46:40.769075 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.769030    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ecebed5f-1f64-4249-aea9-d59934063afb" (UID: "ecebed5f-1f64-4249-aea9-d59934063afb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:46:40.867278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.867232    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-istio-token\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:46:40.867278 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.867270    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecebed5f-1f64-4249-aea9-d59934063afb-local-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:46:40.867573 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.867279    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtsfk\" (UniqueName: \"kubernetes.io/projected/ecebed5f-1f64-4249-aea9-d59934063afb-kube-api-access-xtsfk\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:46:40.867573 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.867304    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-csr-dns-cert\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:46:40.867573 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.867315    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-cacerts\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:46:40.867573 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:40.867323    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecebed5f-1f64-4249-aea9-d59934063afb-istio-kubeconfig\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:46:41.360020 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.359986    2581 generic.go:356] "Generic (PLEG): container finished" podID="ecebed5f-1f64-4249-aea9-d59934063afb" containerID="e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6" exitCode=0
Jul 14 19:46:41.360524 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.360030    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"
Jul 14 19:46:41.360524 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.360072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt" event={"ID":"ecebed5f-1f64-4249-aea9-d59934063afb","Type":"ContainerDied","Data":"e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6"}
Jul 14 19:46:41.360524 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.360121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt" event={"ID":"ecebed5f-1f64-4249-aea9-d59934063afb","Type":"ContainerDied","Data":"b907953f04905a68f7e7942cfe95c83d56a55bf5ea91f7977d34ccdee7a1f9c3"}
Jul 14 19:46:41.360524 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.360141    2581 scope.go:117] "RemoveContainer" containerID="e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6"
Jul 14 19:46:41.361975 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.361944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw" event={"ID":"489ece5a-4af8-4670-bb4a-76119587279d","Type":"ContainerStarted","Data":"78985fca0a311ddea8ed12ad299405efd2c7b50c967b90f7fe49a230fc51fa58"}
Jul 14 19:46:41.362090 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.362069    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:46:41.369630 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.369610    2581 scope.go:117] "RemoveContainer" containerID="e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6"
Jul 14 19:46:41.369917 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:46:41.369890    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6\": container with ID starting with e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6 not found: ID does not exist" containerID="e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6"
Jul 14 19:46:41.369980 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.369927    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6"} err="failed to get container status \"e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6\": rpc error: code = NotFound desc = could not find container \"e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6\": container with ID starting with e925160219c1cdaaf6c1b6bda7d0a69c1ad06f04d13a7045514c94f8dd0467d6 not found: ID does not exist"
Jul 14 19:46:41.488990 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.488923    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw" podStartSLOduration=2.738704158 podStartE2EDuration="6.488897928s" totalImagesPullingTime="3.75019377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:46:41.456108434 +0000 UTC m=+499.767807890" watchObservedRunningTime="2026-07-14 19:46:41.488897928 +0000 UTC m=+499.800597383"
Jul 14 19:46:41.489591 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.489570    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"]
Jul 14 19:46:41.515106 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:41.515074    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5p2jt"]
Jul 14 19:46:43.339321 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:46:43.339285    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecebed5f-1f64-4249-aea9-d59934063afb" path="/var/lib/kubelet/pods/ecebed5f-1f64-4249-aea9-d59934063afb/volumes"
Jul 14 19:47:11.367853 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:11.367814    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jdhc"
Jul 14 19:47:12.371824 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:12.371795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bwjw"
Jul 14 19:47:23.330499 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.330414    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cwb2n"]
Jul 14 19:47:23.330862 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.330771    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecebed5f-1f64-4249-aea9-d59934063afb" containerName="discovery"
Jul 14 19:47:23.330862 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.330783    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecebed5f-1f64-4249-aea9-d59934063afb" containerName="discovery"
Jul 14 19:47:23.330862 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.330854    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecebed5f-1f64-4249-aea9-d59934063afb" containerName="discovery"
Jul 14 19:47:23.335163 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.335132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n"
Jul 14 19:47:23.337597 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.337570    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 19:47:23.338473 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.338445    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-wsxgm\""
Jul 14 19:47:23.338618 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.338447    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 19:47:23.346523 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.346497    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cwb2n"]
Jul 14 19:47:23.462087 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.462053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-848rm\" (UniqueName: \"kubernetes.io/projected/41669ca2-5990-4d10-9ae0-0aecd3728605-kube-api-access-848rm\") pod \"kuadrant-operator-catalog-cwb2n\" (UID: \"41669ca2-5990-4d10-9ae0-0aecd3728605\") " pod="kuadrant-system/kuadrant-operator-catalog-cwb2n"
Jul 14 19:47:23.563557 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.563511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-848rm\" (UniqueName: \"kubernetes.io/projected/41669ca2-5990-4d10-9ae0-0aecd3728605-kube-api-access-848rm\") pod \"kuadrant-operator-catalog-cwb2n\" (UID: \"41669ca2-5990-4d10-9ae0-0aecd3728605\") " pod="kuadrant-system/kuadrant-operator-catalog-cwb2n"
Jul 14 19:47:23.572480 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.572446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-848rm\" (UniqueName: \"kubernetes.io/projected/41669ca2-5990-4d10-9ae0-0aecd3728605-kube-api-access-848rm\") pod \"kuadrant-operator-catalog-cwb2n\" (UID: \"41669ca2-5990-4d10-9ae0-0aecd3728605\") " pod="kuadrant-system/kuadrant-operator-catalog-cwb2n"
Jul 14 19:47:23.648329 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.648231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n"
Jul 14 19:47:23.684042 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.683999    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cwb2n"]
Jul 14 19:47:23.788882 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.788856    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cwb2n"]
Jul 14 19:47:23.790798 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:47:23.790772    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41669ca2_5990_4d10_9ae0_0aecd3728605.slice/crio-c16ab0255a5b49892b93b8f0b1f38252642b1e34b4000f7ccffa85a75dc317dc WatchSource:0}: Error finding container c16ab0255a5b49892b93b8f0b1f38252642b1e34b4000f7ccffa85a75dc317dc: Status 404 returned error can't find the container with id c16ab0255a5b49892b93b8f0b1f38252642b1e34b4000f7ccffa85a75dc317dc
Jul 14 19:47:23.893198 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.893160    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nq2sf"]
Jul 14 19:47:23.896215 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.896194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:23.905049 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.905021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nq2sf"]
Jul 14 19:47:23.966919 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:23.966884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxc5p\" (UniqueName: \"kubernetes.io/projected/1ebbdc46-3c55-4513-ae2b-d11867280462-kube-api-access-dxc5p\") pod \"kuadrant-operator-catalog-nq2sf\" (UID: \"1ebbdc46-3c55-4513-ae2b-d11867280462\") " pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:24.068173 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:24.068128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxc5p\" (UniqueName: \"kubernetes.io/projected/1ebbdc46-3c55-4513-ae2b-d11867280462-kube-api-access-dxc5p\") pod \"kuadrant-operator-catalog-nq2sf\" (UID: \"1ebbdc46-3c55-4513-ae2b-d11867280462\") " pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:24.076808 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:24.076771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxc5p\" (UniqueName: \"kubernetes.io/projected/1ebbdc46-3c55-4513-ae2b-d11867280462-kube-api-access-dxc5p\") pod \"kuadrant-operator-catalog-nq2sf\" (UID: \"1ebbdc46-3c55-4513-ae2b-d11867280462\") " pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:24.207679 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:24.207635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:24.357704 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:24.357527    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nq2sf"]
Jul 14 19:47:24.360240 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:47:24.360214    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ebbdc46_3c55_4513_ae2b_d11867280462.slice/crio-6ce6034b125a96b599591b487da83092207fa67366fd516299444e6d2b52ddef WatchSource:0}: Error finding container 6ce6034b125a96b599591b487da83092207fa67366fd516299444e6d2b52ddef: Status 404 returned error can't find the container with id 6ce6034b125a96b599591b487da83092207fa67366fd516299444e6d2b52ddef
Jul 14 19:47:24.591094 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:24.590996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n" event={"ID":"41669ca2-5990-4d10-9ae0-0aecd3728605","Type":"ContainerStarted","Data":"c16ab0255a5b49892b93b8f0b1f38252642b1e34b4000f7ccffa85a75dc317dc"}
Jul 14 19:47:24.592669 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:24.592635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf" event={"ID":"1ebbdc46-3c55-4513-ae2b-d11867280462","Type":"ContainerStarted","Data":"6ce6034b125a96b599591b487da83092207fa67366fd516299444e6d2b52ddef"}
Jul 14 19:47:26.606130 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.606081    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf" event={"ID":"1ebbdc46-3c55-4513-ae2b-d11867280462","Type":"ContainerStarted","Data":"9da8610aea957729b3bb2517b551fe0d88920252bffce38d12f059a56d3031bd"}
Jul 14 19:47:26.607625 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.607597    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n" event={"ID":"41669ca2-5990-4d10-9ae0-0aecd3728605","Type":"ContainerStarted","Data":"422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99"}
Jul 14 19:47:26.607740 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.607690    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n" podUID="41669ca2-5990-4d10-9ae0-0aecd3728605" containerName="registry-server" containerID="cri-o://422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99" gracePeriod=2
Jul 14 19:47:26.623906 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.623854    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf" podStartSLOduration=2.153744484 podStartE2EDuration="3.623835098s" totalImagesPullingTime="1.470090614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:47:26.62276961 +0000 UTC m=+544.934469104" watchObservedRunningTime="2026-07-14 19:47:26.623835098 +0000 UTC m=+544.935534555"
Jul 14 19:47:26.638616 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.638562    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n" podStartSLOduration=1.599719697 podStartE2EDuration="3.638543028s" totalImagesPullingTime="2.038823331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:47:26.636839397 +0000 UTC m=+544.948538852" watchObservedRunningTime="2026-07-14 19:47:26.638543028 +0000 UTC m=+544.950242484"
Jul 14 19:47:26.872302 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.872230    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n"
Jul 14 19:47:26.993497 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.993454    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-848rm\" (UniqueName: \"kubernetes.io/projected/41669ca2-5990-4d10-9ae0-0aecd3728605-kube-api-access-848rm\") pod \"41669ca2-5990-4d10-9ae0-0aecd3728605\" (UID: \"41669ca2-5990-4d10-9ae0-0aecd3728605\") "
Jul 14 19:47:26.995733 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:26.995702    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41669ca2-5990-4d10-9ae0-0aecd3728605-kube-api-access-848rm" (OuterVolumeSpecName: "kube-api-access-848rm") pod "41669ca2-5990-4d10-9ae0-0aecd3728605" (UID: "41669ca2-5990-4d10-9ae0-0aecd3728605"). InnerVolumeSpecName "kube-api-access-848rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:47:27.094527 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.094490    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-848rm\" (UniqueName: \"kubernetes.io/projected/41669ca2-5990-4d10-9ae0-0aecd3728605-kube-api-access-848rm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:27.614675 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.614573    2581 generic.go:356] "Generic (PLEG): container finished" podID="41669ca2-5990-4d10-9ae0-0aecd3728605" containerID="422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99" exitCode=0
Jul 14 19:47:27.614675 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.614617    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n"
Jul 14 19:47:27.614675 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.614655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n" event={"ID":"41669ca2-5990-4d10-9ae0-0aecd3728605","Type":"ContainerDied","Data":"422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99"}
Jul 14 19:47:27.615207 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.614719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cwb2n" event={"ID":"41669ca2-5990-4d10-9ae0-0aecd3728605","Type":"ContainerDied","Data":"c16ab0255a5b49892b93b8f0b1f38252642b1e34b4000f7ccffa85a75dc317dc"}
Jul 14 19:47:27.615207 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.614745    2581 scope.go:117] "RemoveContainer" containerID="422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99"
Jul 14 19:47:27.624176 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.624156    2581 scope.go:117] "RemoveContainer" containerID="422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99"
Jul 14 19:47:27.624508 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:47:27.624487    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99\": container with ID starting with 422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99 not found: ID does not exist" containerID="422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99"
Jul 14 19:47:27.624563 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.624520    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99"} err="failed to get container status \"422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99\": rpc error: code = NotFound desc = could not find container \"422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99\": container with ID starting with 422a8f23e81180543aad19f4052fa669131615bb94977826842b7a919a859c99 not found: ID does not exist"
Jul 14 19:47:27.631241 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.631216    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cwb2n"]
Jul 14 19:47:27.633208 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:27.633185    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cwb2n"]
Jul 14 19:47:29.340558 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:29.338137    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41669ca2-5990-4d10-9ae0-0aecd3728605" path="/var/lib/kubelet/pods/41669ca2-5990-4d10-9ae0-0aecd3728605/volumes"
Jul 14 19:47:34.208038 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:34.207994    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:34.208595 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:34.208070    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:34.231985 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:34.231957    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:34.673985 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:34.673903    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-nq2sf"
Jul 14 19:47:38.517371 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.517329    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"]
Jul 14 19:47:38.517790 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.517728    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41669ca2-5990-4d10-9ae0-0aecd3728605" containerName="registry-server"
Jul 14 19:47:38.517790 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.517739    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="41669ca2-5990-4d10-9ae0-0aecd3728605" containerName="registry-server"
Jul 14 19:47:38.517872 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.517813    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="41669ca2-5990-4d10-9ae0-0aecd3728605" containerName="registry-server"
Jul 14 19:47:38.522320 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.522299    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.524847 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.524825    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j6wm7\""
Jul 14 19:47:38.530965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.530936    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"]
Jul 14 19:47:38.606587 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.606546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.606787 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.606655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvhwz\" (UniqueName: \"kubernetes.io/projected/138f51aa-632d-4ac7-81a8-b48881da21b7-kube-api-access-wvhwz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.606787 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.606701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.708066 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.708022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvhwz\" (UniqueName: \"kubernetes.io/projected/138f51aa-632d-4ac7-81a8-b48881da21b7-kube-api-access-wvhwz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.708066 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.708077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.708356 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.708131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.708575 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.708557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.708650 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.708629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.718760 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.718734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvhwz\" (UniqueName: \"kubernetes.io/projected/138f51aa-632d-4ac7-81a8-b48881da21b7-kube-api-access-wvhwz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.832722 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.832603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:38.973324 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:38.973284    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"]
Jul 14 19:47:38.977045 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:47:38.977011    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138f51aa_632d_4ac7_81a8_b48881da21b7.slice/crio-b6ce5744980d4df2285ea74585d6320dd8fa98479fbed58c992210adfd4aa9c3 WatchSource:0}: Error finding container b6ce5744980d4df2285ea74585d6320dd8fa98479fbed58c992210adfd4aa9c3: Status 404 returned error can't find the container with id b6ce5744980d4df2285ea74585d6320dd8fa98479fbed58c992210adfd4aa9c3
Jul 14 19:47:39.109572 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.109480    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"]
Jul 14 19:47:39.113282 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.113256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.121742 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.121707    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"]
Jul 14 19:47:39.213537 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.213498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.213732 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.213567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.213732 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.213681    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbxv\" (UniqueName: \"kubernetes.io/projected/245dd48b-7305-4f00-af93-145b077101a3-kube-api-access-txbxv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.315011 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.314960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbxv\" (UniqueName: \"kubernetes.io/projected/245dd48b-7305-4f00-af93-145b077101a3-kube-api-access-txbxv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.315246 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.315058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.315246 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.315105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.315515 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.315492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.315604 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.315531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.323623 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.323592    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbxv\" (UniqueName: \"kubernetes.io/projected/245dd48b-7305-4f00-af93-145b077101a3-kube-api-access-txbxv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.459513 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.459477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:39.515539 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.515509    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"]
Jul 14 19:47:39.520582 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.520376    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.526973 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.526939    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"]
Jul 14 19:47:39.604624 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.604599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"]
Jul 14 19:47:39.606697 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:47:39.606670    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245dd48b_7305_4f00_af93_145b077101a3.slice/crio-9cd75b2ccf725da1a105d92c63b65acb32da3ce2ab2d0edc866be95be288f510 WatchSource:0}: Error finding container 9cd75b2ccf725da1a105d92c63b65acb32da3ce2ab2d0edc866be95be288f510: Status 404 returned error can't find the container with id 9cd75b2ccf725da1a105d92c63b65acb32da3ce2ab2d0edc866be95be288f510
Jul 14 19:47:39.619030 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.619003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.619030 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.619034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.619188 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.619135    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2qfc\" (UniqueName: \"kubernetes.io/projected/83503fe0-1721-43f3-862f-587585d75e50-kube-api-access-l2qfc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.681079 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.681039    2581 generic.go:356] "Generic (PLEG): container finished" podID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerID="a810136a735f64f661ca52735bffee9f90e7aaba4a4e5441dfed384bcb06d533" exitCode=0
Jul 14 19:47:39.681243 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.681129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h" event={"ID":"138f51aa-632d-4ac7-81a8-b48881da21b7","Type":"ContainerDied","Data":"a810136a735f64f661ca52735bffee9f90e7aaba4a4e5441dfed384bcb06d533"}
Jul 14 19:47:39.681243 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.681191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h" event={"ID":"138f51aa-632d-4ac7-81a8-b48881da21b7","Type":"ContainerStarted","Data":"b6ce5744980d4df2285ea74585d6320dd8fa98479fbed58c992210adfd4aa9c3"}
Jul 14 19:47:39.683164 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.683140    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4" event={"ID":"245dd48b-7305-4f00-af93-145b077101a3","Type":"ContainerStarted","Data":"da80d01bfe02a0960a1f72851d91e8cc92aa84beb10e60bd9937c0458e4b9c33"}
Jul 14 19:47:39.683299 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.683191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4" event={"ID":"245dd48b-7305-4f00-af93-145b077101a3","Type":"ContainerStarted","Data":"9cd75b2ccf725da1a105d92c63b65acb32da3ce2ab2d0edc866be95be288f510"}
Jul 14 19:47:39.720782 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.720739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.720981 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.720801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.720981 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.720916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2qfc\" (UniqueName: \"kubernetes.io/projected/83503fe0-1721-43f3-862f-587585d75e50-kube-api-access-l2qfc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.721921 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.721880    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.722193 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.722173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.730558 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.730527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2qfc\" (UniqueName: \"kubernetes.io/projected/83503fe0-1721-43f3-862f-587585d75e50-kube-api-access-l2qfc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.834803 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.834759    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:39.914566 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.914533    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"]
Jul 14 19:47:39.919519 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.919494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:39.929183 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.929112    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"]
Jul 14 19:47:39.984770 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:39.984742    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"]
Jul 14 19:47:39.987428 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:47:39.987379    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83503fe0_1721_43f3_862f_587585d75e50.slice/crio-ec2f6cc529ac101aee11dc8bfc765cb09adb3455ab1e8424e1c817aaf8ec46da WatchSource:0}: Error finding container ec2f6cc529ac101aee11dc8bfc765cb09adb3455ab1e8424e1c817aaf8ec46da: Status 404 returned error can't find the container with id ec2f6cc529ac101aee11dc8bfc765cb09adb3455ab1e8424e1c817aaf8ec46da
Jul 14 19:47:40.023690 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.023656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.023858 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.023714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.023858 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.023821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2hzt\" (UniqueName: \"kubernetes.io/projected/39a5dc7e-5391-4e3a-a123-fded563dc644-kube-api-access-q2hzt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.124346 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.124301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2hzt\" (UniqueName: \"kubernetes.io/projected/39a5dc7e-5391-4e3a-a123-fded563dc644-kube-api-access-q2hzt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.124571 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.124384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.124571 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.124479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.124940 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.124914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.124976 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.124914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.133565 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.133540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2hzt\" (UniqueName: \"kubernetes.io/projected/39a5dc7e-5391-4e3a-a123-fded563dc644-kube-api-access-q2hzt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.232654 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.232615    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:40.375179 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.375152    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"]
Jul 14 19:47:40.393546 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:47:40.393510    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a5dc7e_5391_4e3a_a123_fded563dc644.slice/crio-413c4dcd351f7c6eefa0b6962602efcfc7bbddd0b60cef344479583b1899d289 WatchSource:0}: Error finding container 413c4dcd351f7c6eefa0b6962602efcfc7bbddd0b60cef344479583b1899d289: Status 404 returned error can't find the container with id 413c4dcd351f7c6eefa0b6962602efcfc7bbddd0b60cef344479583b1899d289
Jul 14 19:47:40.690482 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.690443    2581 generic.go:356] "Generic (PLEG): container finished" podID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerID="38fd984e49278a29660f05beee42df56c560a6ed3fe5c944911dd00860db92f1" exitCode=0
Jul 14 19:47:40.690901 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.690578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n" event={"ID":"39a5dc7e-5391-4e3a-a123-fded563dc644","Type":"ContainerDied","Data":"38fd984e49278a29660f05beee42df56c560a6ed3fe5c944911dd00860db92f1"}
Jul 14 19:47:40.690901 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.690640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n" event={"ID":"39a5dc7e-5391-4e3a-a123-fded563dc644","Type":"ContainerStarted","Data":"413c4dcd351f7c6eefa0b6962602efcfc7bbddd0b60cef344479583b1899d289"}
Jul 14 19:47:40.692651 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.692620    2581 generic.go:356] "Generic (PLEG): container finished" podID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerID="496fe06d92f6bd4235ecd4b080b80b990910584a542456b03b34939dc3f567b7" exitCode=0
Jul 14 19:47:40.692763 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.692686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h" event={"ID":"138f51aa-632d-4ac7-81a8-b48881da21b7","Type":"ContainerDied","Data":"496fe06d92f6bd4235ecd4b080b80b990910584a542456b03b34939dc3f567b7"}
Jul 14 19:47:40.694312 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.694290    2581 generic.go:356] "Generic (PLEG): container finished" podID="245dd48b-7305-4f00-af93-145b077101a3" containerID="da80d01bfe02a0960a1f72851d91e8cc92aa84beb10e60bd9937c0458e4b9c33" exitCode=0
Jul 14 19:47:40.694366 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.694354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4" event={"ID":"245dd48b-7305-4f00-af93-145b077101a3","Type":"ContainerDied","Data":"da80d01bfe02a0960a1f72851d91e8cc92aa84beb10e60bd9937c0458e4b9c33"}
Jul 14 19:47:40.696075 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.696051    2581 generic.go:356] "Generic (PLEG): container finished" podID="83503fe0-1721-43f3-862f-587585d75e50" containerID="8ae21b8407a8ec80b16cf8d3771b00ae5eb3ff4f20ce99c6f5bc3e715c3417b1" exitCode=0
Jul 14 19:47:40.696159 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.696111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq" event={"ID":"83503fe0-1721-43f3-862f-587585d75e50","Type":"ContainerDied","Data":"8ae21b8407a8ec80b16cf8d3771b00ae5eb3ff4f20ce99c6f5bc3e715c3417b1"}
Jul 14 19:47:40.696159 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:40.696145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq" event={"ID":"83503fe0-1721-43f3-862f-587585d75e50","Type":"ContainerStarted","Data":"ec2f6cc529ac101aee11dc8bfc765cb09adb3455ab1e8424e1c817aaf8ec46da"}
Jul 14 19:47:41.704815 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:41.704775    2581 generic.go:356] "Generic (PLEG): container finished" podID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerID="84f11f871b14f3d02420d97c61b949378a895135a20b319aa4d467a7c3d49cd0" exitCode=0
Jul 14 19:47:41.705255 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:41.704856    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h" event={"ID":"138f51aa-632d-4ac7-81a8-b48881da21b7","Type":"ContainerDied","Data":"84f11f871b14f3d02420d97c61b949378a895135a20b319aa4d467a7c3d49cd0"}
Jul 14 19:47:41.710665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:41.708737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4" event={"ID":"245dd48b-7305-4f00-af93-145b077101a3","Type":"ContainerStarted","Data":"bb869bf2b464f666f9d53bcadfc6676f39fe80c60fac879405dd9ea0c1e4d668"}
Jul 14 19:47:41.713021 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:41.712994    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n" event={"ID":"39a5dc7e-5391-4e3a-a123-fded563dc644","Type":"ContainerStarted","Data":"c157fcba312471a29ad97376a8d7ee20888eaa5e7d7a60fcde3089a05d911a4d"}
Jul 14 19:47:42.720614 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.720580    2581 generic.go:356] "Generic (PLEG): container finished" podID="83503fe0-1721-43f3-862f-587585d75e50" containerID="95b636c3e141b3510571b3be2eae6f2fd96bc08c10cae676e7b681c41dd05059" exitCode=0
Jul 14 19:47:42.721129 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.720662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq" event={"ID":"83503fe0-1721-43f3-862f-587585d75e50","Type":"ContainerDied","Data":"95b636c3e141b3510571b3be2eae6f2fd96bc08c10cae676e7b681c41dd05059"}
Jul 14 19:47:42.723076 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.723054    2581 generic.go:356] "Generic (PLEG): container finished" podID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerID="c157fcba312471a29ad97376a8d7ee20888eaa5e7d7a60fcde3089a05d911a4d" exitCode=0
Jul 14 19:47:42.723076 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.723073    2581 generic.go:356] "Generic (PLEG): container finished" podID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerID="edcc4a18fca0331f0ed96b0a39279c8a5058c87e5abd53ae453bef458f2fffca" exitCode=0
Jul 14 19:47:42.723234 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.723136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n" event={"ID":"39a5dc7e-5391-4e3a-a123-fded563dc644","Type":"ContainerDied","Data":"c157fcba312471a29ad97376a8d7ee20888eaa5e7d7a60fcde3089a05d911a4d"}
Jul 14 19:47:42.723234 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.723184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n" event={"ID":"39a5dc7e-5391-4e3a-a123-fded563dc644","Type":"ContainerDied","Data":"edcc4a18fca0331f0ed96b0a39279c8a5058c87e5abd53ae453bef458f2fffca"}
Jul 14 19:47:42.725286 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.725243    2581 generic.go:356] "Generic (PLEG): container finished" podID="245dd48b-7305-4f00-af93-145b077101a3" containerID="bb869bf2b464f666f9d53bcadfc6676f39fe80c60fac879405dd9ea0c1e4d668" exitCode=0
Jul 14 19:47:42.725286 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.725262    2581 generic.go:356] "Generic (PLEG): container finished" podID="245dd48b-7305-4f00-af93-145b077101a3" containerID="b56a2cdb666bbd9128a48636b53e0d47b72c1ec15aece376bf162c5a3f98be13" exitCode=0
Jul 14 19:47:42.725440 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.725316    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4" event={"ID":"245dd48b-7305-4f00-af93-145b077101a3","Type":"ContainerDied","Data":"bb869bf2b464f666f9d53bcadfc6676f39fe80c60fac879405dd9ea0c1e4d668"}
Jul 14 19:47:42.725440 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.725352    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4" event={"ID":"245dd48b-7305-4f00-af93-145b077101a3","Type":"ContainerDied","Data":"b56a2cdb666bbd9128a48636b53e0d47b72c1ec15aece376bf162c5a3f98be13"}
Jul 14 19:47:42.875778 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:42.875748    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:43.052702 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.052607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvhwz\" (UniqueName: \"kubernetes.io/projected/138f51aa-632d-4ac7-81a8-b48881da21b7-kube-api-access-wvhwz\") pod \"138f51aa-632d-4ac7-81a8-b48881da21b7\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") "
Jul 14 19:47:43.052702 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.052662    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-bundle\") pod \"138f51aa-632d-4ac7-81a8-b48881da21b7\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") "
Jul 14 19:47:43.052925 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.052765    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-util\") pod \"138f51aa-632d-4ac7-81a8-b48881da21b7\" (UID: \"138f51aa-632d-4ac7-81a8-b48881da21b7\") "
Jul 14 19:47:43.053162 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.053136    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-bundle" (OuterVolumeSpecName: "bundle") pod "138f51aa-632d-4ac7-81a8-b48881da21b7" (UID: "138f51aa-632d-4ac7-81a8-b48881da21b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:43.054895 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.054862    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/138f51aa-632d-4ac7-81a8-b48881da21b7-kube-api-access-wvhwz" (OuterVolumeSpecName: "kube-api-access-wvhwz") pod "138f51aa-632d-4ac7-81a8-b48881da21b7" (UID: "138f51aa-632d-4ac7-81a8-b48881da21b7"). InnerVolumeSpecName "kube-api-access-wvhwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:47:43.061341 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.061314    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-util" (OuterVolumeSpecName: "util") pod "138f51aa-632d-4ac7-81a8-b48881da21b7" (UID: "138f51aa-632d-4ac7-81a8-b48881da21b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:43.154195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.154156    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-util\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:43.154195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.154190    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvhwz\" (UniqueName: \"kubernetes.io/projected/138f51aa-632d-4ac7-81a8-b48881da21b7-kube-api-access-wvhwz\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:43.154195 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.154200    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/138f51aa-632d-4ac7-81a8-b48881da21b7-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:43.732896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.732861    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h"
Jul 14 19:47:43.733352 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.732862    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h" event={"ID":"138f51aa-632d-4ac7-81a8-b48881da21b7","Type":"ContainerDied","Data":"b6ce5744980d4df2285ea74585d6320dd8fa98479fbed58c992210adfd4aa9c3"}
Jul 14 19:47:43.733352 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.733001    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6ce5744980d4df2285ea74585d6320dd8fa98479fbed58c992210adfd4aa9c3"
Jul 14 19:47:43.735305 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.735274    2581 generic.go:356] "Generic (PLEG): container finished" podID="83503fe0-1721-43f3-862f-587585d75e50" containerID="cf7e13417853256c6a1a76b12c2d330debae3fe683b6811acdc3560bd4d1f128" exitCode=0
Jul 14 19:47:43.735470 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.735360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq" event={"ID":"83503fe0-1721-43f3-862f-587585d75e50","Type":"ContainerDied","Data":"cf7e13417853256c6a1a76b12c2d330debae3fe683b6811acdc3560bd4d1f128"}
Jul 14 19:47:43.901339 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.901314    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:43.903553 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:43.903532    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:44.062955 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.062858    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txbxv\" (UniqueName: \"kubernetes.io/projected/245dd48b-7305-4f00-af93-145b077101a3-kube-api-access-txbxv\") pod \"245dd48b-7305-4f00-af93-145b077101a3\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") "
Jul 14 19:47:44.062955 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.062923    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2hzt\" (UniqueName: \"kubernetes.io/projected/39a5dc7e-5391-4e3a-a123-fded563dc644-kube-api-access-q2hzt\") pod \"39a5dc7e-5391-4e3a-a123-fded563dc644\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") "
Jul 14 19:47:44.063194 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.062967    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-util\") pod \"245dd48b-7305-4f00-af93-145b077101a3\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") "
Jul 14 19:47:44.063194 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.063021    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-util\") pod \"39a5dc7e-5391-4e3a-a123-fded563dc644\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") "
Jul 14 19:47:44.063194 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.063060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-bundle\") pod \"245dd48b-7305-4f00-af93-145b077101a3\" (UID: \"245dd48b-7305-4f00-af93-145b077101a3\") "
Jul 14 19:47:44.063194 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.063094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-bundle\") pod \"39a5dc7e-5391-4e3a-a123-fded563dc644\" (UID: \"39a5dc7e-5391-4e3a-a123-fded563dc644\") "
Jul 14 19:47:44.063723 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.063692    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-bundle" (OuterVolumeSpecName: "bundle") pod "245dd48b-7305-4f00-af93-145b077101a3" (UID: "245dd48b-7305-4f00-af93-145b077101a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:44.063812 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.063719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-bundle" (OuterVolumeSpecName: "bundle") pod "39a5dc7e-5391-4e3a-a123-fded563dc644" (UID: "39a5dc7e-5391-4e3a-a123-fded563dc644"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:44.065189 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.065162    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a5dc7e-5391-4e3a-a123-fded563dc644-kube-api-access-q2hzt" (OuterVolumeSpecName: "kube-api-access-q2hzt") pod "39a5dc7e-5391-4e3a-a123-fded563dc644" (UID: "39a5dc7e-5391-4e3a-a123-fded563dc644"). InnerVolumeSpecName "kube-api-access-q2hzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:47:44.065449 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.065419    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245dd48b-7305-4f00-af93-145b077101a3-kube-api-access-txbxv" (OuterVolumeSpecName: "kube-api-access-txbxv") pod "245dd48b-7305-4f00-af93-145b077101a3" (UID: "245dd48b-7305-4f00-af93-145b077101a3"). InnerVolumeSpecName "kube-api-access-txbxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:47:44.069754 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.069730    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-util" (OuterVolumeSpecName: "util") pod "39a5dc7e-5391-4e3a-a123-fded563dc644" (UID: "39a5dc7e-5391-4e3a-a123-fded563dc644"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:44.071556 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.071532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-util" (OuterVolumeSpecName: "util") pod "245dd48b-7305-4f00-af93-145b077101a3" (UID: "245dd48b-7305-4f00-af93-145b077101a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:44.164506 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.164463    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-util\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:44.164506 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.164498    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:44.164506 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.164507    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39a5dc7e-5391-4e3a-a123-fded563dc644-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:44.164506 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.164516    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txbxv\" (UniqueName: \"kubernetes.io/projected/245dd48b-7305-4f00-af93-145b077101a3-kube-api-access-txbxv\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:44.164779 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.164528    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2hzt\" (UniqueName: \"kubernetes.io/projected/39a5dc7e-5391-4e3a-a123-fded563dc644-kube-api-access-q2hzt\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:44.164779 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.164536    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245dd48b-7305-4f00-af93-145b077101a3-util\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:44.745384 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.745341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4" event={"ID":"245dd48b-7305-4f00-af93-145b077101a3","Type":"ContainerDied","Data":"9cd75b2ccf725da1a105d92c63b65acb32da3ce2ab2d0edc866be95be288f510"}
Jul 14 19:47:44.745847 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.745418    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cd75b2ccf725da1a105d92c63b65acb32da3ce2ab2d0edc866be95be288f510"
Jul 14 19:47:44.745847 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.745358    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4"
Jul 14 19:47:44.747506 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.747471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n" event={"ID":"39a5dc7e-5391-4e3a-a123-fded563dc644","Type":"ContainerDied","Data":"413c4dcd351f7c6eefa0b6962602efcfc7bbddd0b60cef344479583b1899d289"}
Jul 14 19:47:44.747506 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.747502    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n"
Jul 14 19:47:44.747659 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.747523    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="413c4dcd351f7c6eefa0b6962602efcfc7bbddd0b60cef344479583b1899d289"
Jul 14 19:47:44.890218 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:44.890188    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:45.074087 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.073986    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-bundle\") pod \"83503fe0-1721-43f3-862f-587585d75e50\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") "
Jul 14 19:47:45.074087 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.074034    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-util\") pod \"83503fe0-1721-43f3-862f-587585d75e50\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") "
Jul 14 19:47:45.074087 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.074083    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2qfc\" (UniqueName: \"kubernetes.io/projected/83503fe0-1721-43f3-862f-587585d75e50-kube-api-access-l2qfc\") pod \"83503fe0-1721-43f3-862f-587585d75e50\" (UID: \"83503fe0-1721-43f3-862f-587585d75e50\") "
Jul 14 19:47:45.074711 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.074683    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-bundle" (OuterVolumeSpecName: "bundle") pod "83503fe0-1721-43f3-862f-587585d75e50" (UID: "83503fe0-1721-43f3-862f-587585d75e50"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:45.076268 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.076245    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83503fe0-1721-43f3-862f-587585d75e50-kube-api-access-l2qfc" (OuterVolumeSpecName: "kube-api-access-l2qfc") pod "83503fe0-1721-43f3-862f-587585d75e50" (UID: "83503fe0-1721-43f3-862f-587585d75e50"). InnerVolumeSpecName "kube-api-access-l2qfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:47:45.079607 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.079578    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-util" (OuterVolumeSpecName: "util") pod "83503fe0-1721-43f3-862f-587585d75e50" (UID: "83503fe0-1721-43f3-862f-587585d75e50"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:47:45.175735 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.175705    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:45.175735 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.175732    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83503fe0-1721-43f3-862f-587585d75e50-util\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:45.175735 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.175742    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2qfc\" (UniqueName: \"kubernetes.io/projected/83503fe0-1721-43f3-862f-587585d75e50-kube-api-access-l2qfc\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:47:45.754748 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.754714    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq"
Jul 14 19:47:45.755146 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.754712    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq" event={"ID":"83503fe0-1721-43f3-862f-587585d75e50","Type":"ContainerDied","Data":"ec2f6cc529ac101aee11dc8bfc765cb09adb3455ab1e8424e1c817aaf8ec46da"}
Jul 14 19:47:45.755146 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:45.754835    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec2f6cc529ac101aee11dc8bfc765cb09adb3455ab1e8424e1c817aaf8ec46da"
Jul 14 19:47:54.721044 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.720995    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"]
Jul 14 19:47:54.721683 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721661    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerName="util"
Jul 14 19:47:54.721785 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721772    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerName="util"
Jul 14 19:47:54.721837 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721793    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83503fe0-1721-43f3-862f-587585d75e50" containerName="util"
Jul 14 19:47:54.721837 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721803    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83503fe0-1721-43f3-862f-587585d75e50" containerName="util"
Jul 14 19:47:54.721837 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721818    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerName="util"
Jul 14 19:47:54.721837 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721827    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerName="util"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721842    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerName="extract"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721851    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerName="extract"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721864    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83503fe0-1721-43f3-862f-587585d75e50" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721872    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83503fe0-1721-43f3-862f-587585d75e50" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721879    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83503fe0-1721-43f3-862f-587585d75e50" containerName="extract"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721884    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83503fe0-1721-43f3-862f-587585d75e50" containerName="extract"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721892    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721896    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721912    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245dd48b-7305-4f00-af93-145b077101a3" containerName="extract"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721918    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="245dd48b-7305-4f00-af93-145b077101a3" containerName="extract"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721931    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721936    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721954    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245dd48b-7305-4f00-af93-145b077101a3" containerName="util"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721959    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="245dd48b-7305-4f00-af93-145b077101a3" containerName="util"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721965    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245dd48b-7305-4f00-af93-145b077101a3" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721969    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="245dd48b-7305-4f00-af93-145b077101a3" containerName="pull"
Jul 14 19:47:54.721971 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721976    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerName="extract"
Jul 14 19:47:54.722492 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.721981    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerName="extract"
Jul 14 19:47:54.722492 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.722059    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="83503fe0-1721-43f3-862f-587585d75e50" containerName="extract"
Jul 14 19:47:54.722492 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.722067    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="138f51aa-632d-4ac7-81a8-b48881da21b7" containerName="extract"
Jul 14 19:47:54.722492 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.722078    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="245dd48b-7305-4f00-af93-145b077101a3" containerName="extract"
Jul 14 19:47:54.722492 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.722084    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a5dc7e-5391-4e3a-a123-fded563dc644" containerName="extract"
Jul 14 19:47:54.725383 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.725363    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:47:54.728200 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.728172    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t5xdc\""
Jul 14 19:47:54.747869 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.747835    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"]
Jul 14 19:47:54.765188 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.765155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4vlm\" (UniqueName: \"kubernetes.io/projected/6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4-kube-api-access-d4vlm\") pod \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" (UID: \"6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:47:54.865791 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.865749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4vlm\" (UniqueName: \"kubernetes.io/projected/6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4-kube-api-access-d4vlm\") pod \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" (UID: \"6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:47:54.878665 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:54.878628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4vlm\" (UniqueName: \"kubernetes.io/projected/6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4-kube-api-access-d4vlm\") pod \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" (UID: \"6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:47:55.036677 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:55.036577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:47:55.189135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:55.189109    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"]
Jul 14 19:47:55.191644 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:47:55.191614    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5eff7b_b83e_4c9c_8dcb_fb8f31f594e4.slice/crio-60898525cd865e317353e21fbe7e68b81ea8df15b3d1a0548b207d9b8ba4f2b6 WatchSource:0}: Error finding container 60898525cd865e317353e21fbe7e68b81ea8df15b3d1a0548b207d9b8ba4f2b6: Status 404 returned error can't find the container with id 60898525cd865e317353e21fbe7e68b81ea8df15b3d1a0548b207d9b8ba4f2b6
Jul 14 19:47:55.810537 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:55.810497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk" event={"ID":"6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4","Type":"ContainerStarted","Data":"60898525cd865e317353e21fbe7e68b81ea8df15b3d1a0548b207d9b8ba4f2b6"}
Jul 14 19:47:57.823288 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:57.823246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk" event={"ID":"6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4","Type":"ContainerStarted","Data":"dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5"}
Jul 14 19:47:57.823689 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:57.823437    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:47:57.842708 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:57.842650    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk" podStartSLOduration=1.696774391 podStartE2EDuration="3.842634571s" totalImagesPullingTime="2.14586018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:47:57.840806988 +0000 UTC m=+576.152506463" watchObservedRunningTime="2026-07-14 19:47:57.842634571 +0000 UTC m=+576.154334026"
Jul 14 19:47:59.756634 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.756589    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"]
Jul 14 19:47:59.765084 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.765053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:47:59.767644 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.767623    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8tl8d\""
Jul 14 19:47:59.772934 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.772902    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"]
Jul 14 19:47:59.819874 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.819836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw6cm\" (UniqueName: \"kubernetes.io/projected/af57ada2-5327-4a0e-92b2-1950c3a23f86-kube-api-access-sw6cm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:47:59.820069 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.819949    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af57ada2-5327-4a0e-92b2-1950c3a23f86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:47:59.920782 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.920719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af57ada2-5327-4a0e-92b2-1950c3a23f86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:47:59.920958 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.920834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw6cm\" (UniqueName: \"kubernetes.io/projected/af57ada2-5327-4a0e-92b2-1950c3a23f86-kube-api-access-sw6cm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:47:59.921149 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.921128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af57ada2-5327-4a0e-92b2-1950c3a23f86-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:47:59.932153 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:47:59.932119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw6cm\" (UniqueName: \"kubernetes.io/projected/af57ada2-5327-4a0e-92b2-1950c3a23f86-kube-api-access-sw6cm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:00.077193 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:00.077096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:00.228789 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:00.228757    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"]
Jul 14 19:48:00.230907 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:48:00.230867    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf57ada2_5327_4a0e_92b2_1950c3a23f86.slice/crio-05086ec29ffc8c2a4c5018edad6e8f5f808073217ff4a001271dc8cc1e291b2e WatchSource:0}: Error finding container 05086ec29ffc8c2a4c5018edad6e8f5f808073217ff4a001271dc8cc1e291b2e: Status 404 returned error can't find the container with id 05086ec29ffc8c2a4c5018edad6e8f5f808073217ff4a001271dc8cc1e291b2e
Jul 14 19:48:00.842836 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:00.842798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8" event={"ID":"af57ada2-5327-4a0e-92b2-1950c3a23f86","Type":"ContainerStarted","Data":"05086ec29ffc8c2a4c5018edad6e8f5f808073217ff4a001271dc8cc1e291b2e"}
Jul 14 19:48:02.954528 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:02.954493    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"]
Jul 14 19:48:02.958211 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:02.958191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"
Jul 14 19:48:02.961041 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:02.961020    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 14 19:48:02.961462 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:02.961448    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-gw6pw\""
Jul 14 19:48:02.982914 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:02.982818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"]
Jul 14 19:48:03.047045 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:03.047004    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72rpk\" (UniqueName: \"kubernetes.io/projected/741366f7-0b23-4447-b9da-9ce6f424b92c-kube-api-access-72rpk\") pod \"dns-operator-controller-manager-864dd78c4c-wchgh\" (UID: \"741366f7-0b23-4447-b9da-9ce6f424b92c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"
Jul 14 19:48:03.148065 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:03.148020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72rpk\" (UniqueName: \"kubernetes.io/projected/741366f7-0b23-4447-b9da-9ce6f424b92c-kube-api-access-72rpk\") pod \"dns-operator-controller-manager-864dd78c4c-wchgh\" (UID: \"741366f7-0b23-4447-b9da-9ce6f424b92c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"
Jul 14 19:48:03.159135 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:03.159101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72rpk\" (UniqueName: \"kubernetes.io/projected/741366f7-0b23-4447-b9da-9ce6f424b92c-kube-api-access-72rpk\") pod \"dns-operator-controller-manager-864dd78c4c-wchgh\" (UID: \"741366f7-0b23-4447-b9da-9ce6f424b92c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"
Jul 14 19:48:03.269924 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:03.269778    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"
Jul 14 19:48:03.521829 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:03.521796    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"]
Jul 14 19:48:03.524781 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:48:03.524431    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741366f7_0b23_4447_b9da_9ce6f424b92c.slice/crio-5af08cdab3e84a8d52fdeb76edc8d23c8c60da77abd3b3350f2d981e04f5f006 WatchSource:0}: Error finding container 5af08cdab3e84a8d52fdeb76edc8d23c8c60da77abd3b3350f2d981e04f5f006: Status 404 returned error can't find the container with id 5af08cdab3e84a8d52fdeb76edc8d23c8c60da77abd3b3350f2d981e04f5f006
Jul 14 19:48:03.864483 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:03.864329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh" event={"ID":"741366f7-0b23-4447-b9da-9ce6f424b92c","Type":"ContainerStarted","Data":"5af08cdab3e84a8d52fdeb76edc8d23c8c60da77abd3b3350f2d981e04f5f006"}
Jul 14 19:48:05.880764 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:05.880693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh" event={"ID":"741366f7-0b23-4447-b9da-9ce6f424b92c","Type":"ContainerStarted","Data":"3e207ef3c252ccfe4ec1f40538c874cb73264f695abeafd586dc3abb2e54df97"}
Jul 14 19:48:05.880764 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:05.880763    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"
Jul 14 19:48:05.905250 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:05.905192    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh" podStartSLOduration=1.722813941 podStartE2EDuration="3.905168547s" totalImagesPullingTime="2.182354606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:05.90229164 +0000 UTC m=+584.213991096" watchObservedRunningTime="2026-07-14 19:48:05.905168547 +0000 UTC m=+584.216868002"
Jul 14 19:48:06.449884 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.449825    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"]
Jul 14 19:48:06.456018 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.455988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"
Jul 14 19:48:06.458530 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.458506    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-r8b64\""
Jul 14 19:48:06.464884 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.464750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"]
Jul 14 19:48:06.586360 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.586319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8ft\" (UniqueName: \"kubernetes.io/projected/7366817a-8439-43ca-b6d5-a5b8dfae5437-kube-api-access-vw8ft\") pod \"authorino-operator-75cc8d5cb7-5tcss\" (UID: \"7366817a-8439-43ca-b6d5-a5b8dfae5437\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"
Jul 14 19:48:06.687919 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.687877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8ft\" (UniqueName: \"kubernetes.io/projected/7366817a-8439-43ca-b6d5-a5b8dfae5437-kube-api-access-vw8ft\") pod \"authorino-operator-75cc8d5cb7-5tcss\" (UID: \"7366817a-8439-43ca-b6d5-a5b8dfae5437\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"
Jul 14 19:48:06.697133 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.697099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8ft\" (UniqueName: \"kubernetes.io/projected/7366817a-8439-43ca-b6d5-a5b8dfae5437-kube-api-access-vw8ft\") pod \"authorino-operator-75cc8d5cb7-5tcss\" (UID: \"7366817a-8439-43ca-b6d5-a5b8dfae5437\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"
Jul 14 19:48:06.769125 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.769027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"
Jul 14 19:48:06.929017 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:06.928990    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"]
Jul 14 19:48:06.930522 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:48:06.930475    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7366817a_8439_43ca_b6d5_a5b8dfae5437.slice/crio-b6d804a1e72be72baf45cd125ffc9a945e2cfacab5a9131850203268e30f6788 WatchSource:0}: Error finding container b6d804a1e72be72baf45cd125ffc9a945e2cfacab5a9131850203268e30f6788: Status 404 returned error can't find the container with id b6d804a1e72be72baf45cd125ffc9a945e2cfacab5a9131850203268e30f6788
Jul 14 19:48:07.898764 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:07.898720    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss" event={"ID":"7366817a-8439-43ca-b6d5-a5b8dfae5437","Type":"ContainerStarted","Data":"b6d804a1e72be72baf45cd125ffc9a945e2cfacab5a9131850203268e30f6788"}
Jul 14 19:48:08.832696 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:08.832656    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:10.917848 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:10.917807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8" event={"ID":"af57ada2-5327-4a0e-92b2-1950c3a23f86","Type":"ContainerStarted","Data":"b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402"}
Jul 14 19:48:10.918332 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:10.917885    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:10.919457 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:10.919429    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss" event={"ID":"7366817a-8439-43ca-b6d5-a5b8dfae5437","Type":"ContainerStarted","Data":"cd7b86a6f3c10ccd7dac4fc36eb65519ef49b3980b1945eef5a7b20e48879dc2"}
Jul 14 19:48:10.919582 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:10.919525    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"
Jul 14 19:48:10.939840 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:10.939786    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8" podStartSLOduration=1.982824124 podStartE2EDuration="11.939769249s" totalImagesPullingTime="9.956945125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:10.938895449 +0000 UTC m=+589.250594904" watchObservedRunningTime="2026-07-14 19:48:10.939769249 +0000 UTC m=+589.251468706"
Jul 14 19:48:10.957123 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:10.957067    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss" podStartSLOduration=1.6627892530000001 podStartE2EDuration="4.957049059s" totalImagesPullingTime="3.294259806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:10.95639255 +0000 UTC m=+589.268092020" watchObservedRunningTime="2026-07-14 19:48:10.957049059 +0000 UTC m=+589.268748516"
Jul 14 19:48:16.891426 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:16.891369    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wchgh"
Jul 14 19:48:21.934893 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:21.934857    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5tcss"
Jul 14 19:48:21.935349 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:21.934939    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:23.208755 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:23.208717    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:48:23.209372 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:23.209339    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:48:23.980965 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:23.980930    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"]
Jul 14 19:48:23.981183 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:23.981161    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" containerName="manager" containerID="cri-o://b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402" gracePeriod=2
Jul 14 19:48:24.000283 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.000249    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"]
Jul 14 19:48:24.003256 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.003213    2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"af57ada2-5327-4a0e-92b2-1950c3a23f86\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\": pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:24.014981 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.014949    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"]
Jul 14 19:48:24.015190 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.015164    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" containerName="manager" containerID="cri-o://dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5" gracePeriod=2
Jul 14 19:48:24.024093 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.024041    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"]
Jul 14 19:48:24.024977 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.024697    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" containerName="manager"
Jul 14 19:48:24.024977 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.024723    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" containerName="manager"
Jul 14 19:48:24.024977 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.024872    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" containerName="manager"
Jul 14 19:48:24.028573 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.028548    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"]
Jul 14 19:48:24.028705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.028682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.030110 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.030036    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:24.032851 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.032809    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:24.034759 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.034729    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:24.047514 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.047482    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"]
Jul 14 19:48:24.050963 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.050933    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"]
Jul 14 19:48:24.051432 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.051415    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" containerName="manager"
Jul 14 19:48:24.051548 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.051434    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" containerName="manager"
Jul 14 19:48:24.051548 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.051541    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" containerName="manager"
Jul 14 19:48:24.054729 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.054705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"
Jul 14 19:48:24.064347 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.064300    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"]
Jul 14 19:48:24.066895 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.066863    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:24.068722 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.068682    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:24.152619 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.152586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tlg7n\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.152793 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.152632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzx7s\" (UniqueName: \"kubernetes.io/projected/4c26d98e-8629-4927-957d-a4cf3d43a64f-kube-api-access-mzx7s\") pod \"limitador-operator-controller-manager-859d7b78bd-r7s6h\" (UID: \"4c26d98e-8629-4927-957d-a4cf3d43a64f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"
Jul 14 19:48:24.152793 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.152688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc45s\" (UniqueName: \"kubernetes.io/projected/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-kube-api-access-gc45s\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tlg7n\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.254069 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.254033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tlg7n\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.254489 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.254090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzx7s\" (UniqueName: \"kubernetes.io/projected/4c26d98e-8629-4927-957d-a4cf3d43a64f-kube-api-access-mzx7s\") pod \"limitador-operator-controller-manager-859d7b78bd-r7s6h\" (UID: \"4c26d98e-8629-4927-957d-a4cf3d43a64f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"
Jul 14 19:48:24.254489 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.254144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc45s\" (UniqueName: \"kubernetes.io/projected/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-kube-api-access-gc45s\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tlg7n\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.254489 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.254460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tlg7n\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.274627 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.274593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc45s\" (UniqueName: \"kubernetes.io/projected/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-kube-api-access-gc45s\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tlg7n\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.274804 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.274593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzx7s\" (UniqueName: \"kubernetes.io/projected/4c26d98e-8629-4927-957d-a4cf3d43a64f-kube-api-access-mzx7s\") pod \"limitador-operator-controller-manager-859d7b78bd-r7s6h\" (UID: \"4c26d98e-8629-4927-957d-a4cf3d43a64f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"
Jul 14 19:48:24.276361 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.276330    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:24.278688 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.278648    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:24.279640 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.279620    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:24.280746 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.280724    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:24.282648 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.282617    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:24.284280 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:24.284258    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:24.408692 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.408573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:24.428426 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.414464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"
Jul 14 19:48:24.455750 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.455706    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4vlm\" (UniqueName: \"kubernetes.io/projected/6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4-kube-api-access-d4vlm\") pod \"6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4\" (UID: \"6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4\") "
Jul 14 19:48:24.455948 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.455785    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af57ada2-5327-4a0e-92b2-1950c3a23f86-extensions-socket-volume\") pod \"af57ada2-5327-4a0e-92b2-1950c3a23f86\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") "
Jul 14 19:48:24.455948 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.455856    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw6cm\" (UniqueName: \"kubernetes.io/projected/af57ada2-5327-4a0e-92b2-1950c3a23f86-kube-api-access-sw6cm\") pod \"af57ada2-5327-4a0e-92b2-1950c3a23f86\" (UID: \"af57ada2-5327-4a0e-92b2-1950c3a23f86\") "
Jul 14 19:48:24.456284 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.456250    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af57ada2-5327-4a0e-92b2-1950c3a23f86-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af57ada2-5327-4a0e-92b2-1950c3a23f86" (UID: "af57ada2-5327-4a0e-92b2-1950c3a23f86"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:48:24.459385 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.459340    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4-kube-api-access-d4vlm" (OuterVolumeSpecName: "kube-api-access-d4vlm") pod "6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" (UID: "6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4"). InnerVolumeSpecName "kube-api-access-d4vlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:48:24.460214 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.460187    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af57ada2-5327-4a0e-92b2-1950c3a23f86-kube-api-access-sw6cm" (OuterVolumeSpecName: "kube-api-access-sw6cm") pod "af57ada2-5327-4a0e-92b2-1950c3a23f86" (UID: "af57ada2-5327-4a0e-92b2-1950c3a23f86"). InnerVolumeSpecName "kube-api-access-sw6cm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:48:24.557209 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.557167    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4vlm\" (UniqueName: \"kubernetes.io/projected/6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4-kube-api-access-d4vlm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:48:24.557209 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.557208    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af57ada2-5327-4a0e-92b2-1950c3a23f86-extensions-socket-volume\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:48:24.557507 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.557222    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw6cm\" (UniqueName: \"kubernetes.io/projected/af57ada2-5327-4a0e-92b2-1950c3a23f86-kube-api-access-sw6cm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:48:24.571740 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.571710    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"]
Jul 14 19:48:24.573770 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:48:24.573744    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c86dd5_cbe4_46fc_ba9f_1456ba628b32.slice/crio-3c4d8f2fb7ee9bcb0347848005b362878aa9131455cb3f875131b9e16f1c0c48 WatchSource:0}: Error finding container 3c4d8f2fb7ee9bcb0347848005b362878aa9131455cb3f875131b9e16f1c0c48: Status 404 returned error can't find the container with id 3c4d8f2fb7ee9bcb0347848005b362878aa9131455cb3f875131b9e16f1c0c48
Jul 14 19:48:24.603430 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:24.603383    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"]
Jul 14 19:48:24.615861 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:48:24.615834    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c26d98e_8629_4927_957d_a4cf3d43a64f.slice/crio-31a6c590a6255edaa3b6deb6404e2930fb6285b12dfa6ffe2844182cbe528ca1 WatchSource:0}: Error finding container 31a6c590a6255edaa3b6deb6404e2930fb6285b12dfa6ffe2844182cbe528ca1: Status 404 returned error can't find the container with id 31a6c590a6255edaa3b6deb6404e2930fb6285b12dfa6ffe2844182cbe528ca1
Jul 14 19:48:25.013197 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.013162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h" event={"ID":"4c26d98e-8629-4927-957d-a4cf3d43a64f","Type":"ContainerStarted","Data":"12f0dfaf94b4c263916b46ee939b2d0171e5bfab71f08e9418f6c763555e5725"}
Jul 14 19:48:25.013417 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.013222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h" event={"ID":"4c26d98e-8629-4927-957d-a4cf3d43a64f","Type":"ContainerStarted","Data":"31a6c590a6255edaa3b6deb6404e2930fb6285b12dfa6ffe2844182cbe528ca1"}
Jul 14 19:48:25.013417 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.013338    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"
Jul 14 19:48:25.014573 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.014541    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" containerID="dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5" exitCode=0
Jul 14 19:48:25.014700 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.014591    2581 scope.go:117] "RemoveContainer" containerID="dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5"
Jul 14 19:48:25.014700 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.014593    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:25.015702 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:25.015564    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:25.016139 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.016116    2581 generic.go:356] "Generic (PLEG): container finished" podID="af57ada2-5327-4a0e-92b2-1950c3a23f86" containerID="b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402" exitCode=0
Jul 14 19:48:25.016230 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.016166    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:25.017366 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:25.017298    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:25.018231 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.018161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n" event={"ID":"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32","Type":"ContainerStarted","Data":"bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101"}
Jul 14 19:48:25.018231 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.018226    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:25.018384 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.018252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n" event={"ID":"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32","Type":"ContainerStarted","Data":"3c4d8f2fb7ee9bcb0347848005b362878aa9131455cb3f875131b9e16f1c0c48"}
Jul 14 19:48:25.026057 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.026038    2581 scope.go:117] "RemoveContainer" containerID="dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5"
Jul 14 19:48:25.026384 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:25.026361    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5\": container with ID starting with dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5 not found: ID does not exist" containerID="dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5"
Jul 14 19:48:25.026496 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.026421    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5"} err="failed to get container status \"dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5\": rpc error: code = NotFound desc = could not find container \"dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5\": container with ID starting with dd36e55ef35c0e977db7d628026409b65611ee3e3f6914a82659cb4e812964f5 not found: ID does not exist"
Jul 14 19:48:25.026496 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.026451    2581 scope.go:117] "RemoveContainer" containerID="b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402"
Jul 14 19:48:25.035332 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.035263    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h" podStartSLOduration=1.035244154 podStartE2EDuration="1.035244154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:25.033244701 +0000 UTC m=+603.344944161" watchObservedRunningTime="2026-07-14 19:48:25.035244154 +0000 UTC m=+603.346943609"
Jul 14 19:48:25.036781 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.036745    2581 scope.go:117] "RemoveContainer" containerID="b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402"
Jul 14 19:48:25.037126 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:25.037104    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402\": container with ID starting with b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402 not found: ID does not exist" containerID="b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402"
Jul 14 19:48:25.037190 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.037135    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402"} err="failed to get container status \"b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402\": rpc error: code = NotFound desc = could not find container \"b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402\": container with ID starting with b45a6be93ef306616dfe4868bbf4a908eb761f4bb1d53465de32f6c4df750402 not found: ID does not exist"
Jul 14 19:48:25.055275 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.055216    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n" podStartSLOduration=2.055200806 podStartE2EDuration="2.055200806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:25.054497638 +0000 UTC m=+603.366197094" watchObservedRunningTime="2026-07-14 19:48:25.055200806 +0000 UTC m=+603.366900261"
Jul 14 19:48:25.057224 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:25.057186    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pzhkk\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pzhkk"
Jul 14 19:48:25.059057 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:25.059012    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7k8z8\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k8z8"
Jul 14 19:48:25.339455 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.339340    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4" path="/var/lib/kubelet/pods/6a5eff7b-b83e-4c9c-8dcb-fb8f31f594e4/volumes"
Jul 14 19:48:25.339953 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:25.339912    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af57ada2-5327-4a0e-92b2-1950c3a23f86" path="/var/lib/kubelet/pods/af57ada2-5327-4a0e-92b2-1950c3a23f86/volumes"
Jul 14 19:48:36.029031 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:36.028998    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:36.029464 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:36.029086    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r7s6h"
Jul 14 19:48:40.697721 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.697684    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"]
Jul 14 19:48:40.698163 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.697879    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n" podUID="a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" containerName="manager" containerID="cri-o://bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101" gracePeriod=10
Jul 14 19:48:40.930049 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.930008    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"]
Jul 14 19:48:40.934610 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.934579    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:40.949830 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.949800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"]
Jul 14 19:48:40.967803 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.967776    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:40.996132 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.996091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/80681db0-5c90-45ae-a162-ddc8a14f833e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xzhpz\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:40.996304 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:40.996223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x22qq\" (UniqueName: \"kubernetes.io/projected/80681db0-5c90-45ae-a162-ddc8a14f833e-kube-api-access-x22qq\") pod \"kuadrant-operator-controller-manager-86878c769f-xzhpz\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:41.096671 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.096635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc45s\" (UniqueName: \"kubernetes.io/projected/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-kube-api-access-gc45s\") pod \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") "
Jul 14 19:48:41.096671 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.096679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-extensions-socket-volume\") pod \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\" (UID: \"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32\") "
Jul 14 19:48:41.096941 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.096836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x22qq\" (UniqueName: \"kubernetes.io/projected/80681db0-5c90-45ae-a162-ddc8a14f833e-kube-api-access-x22qq\") pod \"kuadrant-operator-controller-manager-86878c769f-xzhpz\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:41.096941 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.096885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/80681db0-5c90-45ae-a162-ddc8a14f833e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xzhpz\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:41.097163 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.097125    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" (UID: "a8c86dd5-cbe4-46fc-ba9f-1456ba628b32"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:48:41.097297 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.097256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/80681db0-5c90-45ae-a162-ddc8a14f833e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xzhpz\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:41.098925 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.098894    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-kube-api-access-gc45s" (OuterVolumeSpecName: "kube-api-access-gc45s") pod "a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" (UID: "a8c86dd5-cbe4-46fc-ba9f-1456ba628b32"). InnerVolumeSpecName "kube-api-access-gc45s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:48:41.105691 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.105660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x22qq\" (UniqueName: \"kubernetes.io/projected/80681db0-5c90-45ae-a162-ddc8a14f833e-kube-api-access-x22qq\") pod \"kuadrant-operator-controller-manager-86878c769f-xzhpz\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:41.117386 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.117359    2581 generic.go:356] "Generic (PLEG): container finished" podID="a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" containerID="bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101" exitCode=0
Jul 14 19:48:41.117620 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.117427    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"
Jul 14 19:48:41.117620 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.117451    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n" event={"ID":"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32","Type":"ContainerDied","Data":"bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101"}
Jul 14 19:48:41.117620 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.117523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n" event={"ID":"a8c86dd5-cbe4-46fc-ba9f-1456ba628b32","Type":"ContainerDied","Data":"3c4d8f2fb7ee9bcb0347848005b362878aa9131455cb3f875131b9e16f1c0c48"}
Jul 14 19:48:41.117620 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.117558    2581 scope.go:117] "RemoveContainer" containerID="bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101"
Jul 14 19:48:41.127311 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.127287    2581 scope.go:117] "RemoveContainer" containerID="bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101"
Jul 14 19:48:41.127663 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:48:41.127640    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101\": container with ID starting with bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101 not found: ID does not exist" containerID="bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101"
Jul 14 19:48:41.127731 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.127674    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101"} err="failed to get container status \"bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101\": rpc error: code = NotFound desc = could not find container \"bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101\": container with ID starting with bf103b8419015c0ed915615105bd47a4faed3ab168ffea64e46f3d78f6b6b101 not found: ID does not exist"
Jul 14 19:48:41.147476 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.147430    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"]
Jul 14 19:48:41.164154 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.164119    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tlg7n"]
Jul 14 19:48:41.197363 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.197334    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc45s\" (UniqueName: \"kubernetes.io/projected/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-kube-api-access-gc45s\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:48:41.197363 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.197362    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32-extensions-socket-volume\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:48:41.262961 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.262865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:41.338264 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.338231    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" path="/var/lib/kubelet/pods/a8c86dd5-cbe4-46fc-ba9f-1456ba628b32/volumes"
Jul 14 19:48:41.411155 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:41.411129    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"]
Jul 14 19:48:41.413184 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:48:41.413155    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80681db0_5c90_45ae_a162_ddc8a14f833e.slice/crio-f73b3ee89f6dcbefc66174ad773bc13dd3e0656996e5c94d70eff9c10b14915e WatchSource:0}: Error finding container f73b3ee89f6dcbefc66174ad773bc13dd3e0656996e5c94d70eff9c10b14915e: Status 404 returned error can't find the container with id f73b3ee89f6dcbefc66174ad773bc13dd3e0656996e5c94d70eff9c10b14915e
Jul 14 19:48:42.126163 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:42.126125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz" event={"ID":"80681db0-5c90-45ae-a162-ddc8a14f833e","Type":"ContainerStarted","Data":"0b0297c32cf62df1945cdaa95352372090d6b61db83ca2a2181b40c32f940a6d"}
Jul 14 19:48:42.126700 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:42.126182    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz" event={"ID":"80681db0-5c90-45ae-a162-ddc8a14f833e","Type":"ContainerStarted","Data":"f73b3ee89f6dcbefc66174ad773bc13dd3e0656996e5c94d70eff9c10b14915e"}
Jul 14 19:48:42.126700 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:42.126231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:42.150180 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:42.150106    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz" podStartSLOduration=2.150076142 podStartE2EDuration="2.150076142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:42.145569822 +0000 UTC m=+620.457269314" watchObservedRunningTime="2026-07-14 19:48:42.150076142 +0000 UTC m=+620.461775603"
Jul 14 19:48:53.134527 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:53.134494    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:48:57.211739 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.211698    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"]
Jul 14 19:48:57.212280 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.212260    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" containerName="manager"
Jul 14 19:48:57.212280 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.212282    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" containerName="manager"
Jul 14 19:48:57.212377 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.212358    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8c86dd5-cbe4-46fc-ba9f-1456ba628b32" containerName="manager"
Jul 14 19:48:57.215723 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.215702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.218288 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.218265    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5mn5v\""
Jul 14 19:48:57.227528 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.227487    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"]
Jul 14 19:48:57.347669 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.347896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.347896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.347896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s662v\" (UniqueName: \"kubernetes.io/projected/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-kube-api-access-s662v\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.347896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347829    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.347896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.347896 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.348124 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.347924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.348124 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.348059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449177 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449376 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449376 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449533 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449533 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449646 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449817 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449923 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.449923 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.450013 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s662v\" (UniqueName: \"kubernetes.io/projected/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-kube-api-access-s662v\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.450013 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.449998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.450115 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.450035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.450454 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.450428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.450454 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.450445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.452913 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.452887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.453126 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.453102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.458155 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.458119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.458323 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.458302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s662v\" (UniqueName: \"kubernetes.io/projected/18ad6f09-f21e-4cb1-b810-a03ca4a4b180-kube-api-access-s662v\") pod \"maas-default-gateway-openshift-default-56654dcf7-thcwq\" (UID: \"18ad6f09-f21e-4cb1-b810-a03ca4a4b180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.528563 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.528470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:57.678693 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.678666    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"]
Jul 14 19:48:57.681711 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:48:57.681678    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ad6f09_f21e_4cb1_b810_a03ca4a4b180.slice/crio-cc8e0ca8fb4e47c77e9d96d82d7825cc9d3cc4b74857dd13927a3ab8f67c9030 WatchSource:0}: Error finding container cc8e0ca8fb4e47c77e9d96d82d7825cc9d3cc4b74857dd13927a3ab8f67c9030: Status 404 returned error can't find the container with id cc8e0ca8fb4e47c77e9d96d82d7825cc9d3cc4b74857dd13927a3ab8f67c9030
Jul 14 19:48:57.684028 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.683991    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:48:57.684156 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.684081    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:48:57.684156 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:57.684134    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:48:58.224594 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:58.224555    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" event={"ID":"18ad6f09-f21e-4cb1-b810-a03ca4a4b180","Type":"ContainerStarted","Data":"05336926d1509d97b7e8b8160ec2aca5fc21ea602e0fcc440bad72425d001177"}
Jul 14 19:48:58.225005 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:58.224609    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" event={"ID":"18ad6f09-f21e-4cb1-b810-a03ca4a4b180","Type":"ContainerStarted","Data":"cc8e0ca8fb4e47c77e9d96d82d7825cc9d3cc4b74857dd13927a3ab8f67c9030"}
Jul 14 19:48:58.247559 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:58.247503    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" podStartSLOduration=1.247485263 podStartE2EDuration="1.247485263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:48:58.244528943 +0000 UTC m=+636.556228400" watchObservedRunningTime="2026-07-14 19:48:58.247485263 +0000 UTC m=+636.559184718"
Jul 14 19:48:58.529053 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:58.528955    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:58.534279 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:58.534251    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:59.231070 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:59.231042    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:48:59.232189 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:48:59.232167    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:49:04.489196 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.489133    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b54df4f4d-5xczm"]
Jul 14 19:49:04.493455 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.493423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.503198 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.503101    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b54df4f4d-5xczm"]
Jul 14 19:49:04.618999 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.618957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-oauth-serving-cert\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.618999 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.619011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-serving-cert\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.619240 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.619095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-trusted-ca-bundle\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.619240 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.619183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7c4p\" (UniqueName: \"kubernetes.io/projected/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-kube-api-access-l7c4p\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.619240 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.619203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-service-ca\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.619348 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.619239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-config\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.619348 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.619306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-oauth-config\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.719872 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.719836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-oauth-serving-cert\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720068 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.719886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-serving-cert\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720068 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.719928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-trusted-ca-bundle\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720068 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.719948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7c4p\" (UniqueName: \"kubernetes.io/projected/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-kube-api-access-l7c4p\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720068 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.719965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-service-ca\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720068 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.720013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-config\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720354 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.720074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-oauth-config\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720869 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.720833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-oauth-serving-cert\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720989 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.720887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-service-ca\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.720989 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.720939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-config\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.721079 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.721039    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-trusted-ca-bundle\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.722703 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.722672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-oauth-config\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.722805 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.722787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-console-serving-cert\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.731020 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.730984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7c4p\" (UniqueName: \"kubernetes.io/projected/7d5a6d6d-4c30-4488-aed6-1a5ad257602c-kube-api-access-l7c4p\") pod \"console-7b54df4f4d-5xczm\" (UID: \"7d5a6d6d-4c30-4488-aed6-1a5ad257602c\") " pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.805747 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.805660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:04.960070 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:04.960039    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b54df4f4d-5xczm"]
Jul 14 19:49:04.961863 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:49:04.961835    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5a6d6d_4c30_4488_aed6_1a5ad257602c.slice/crio-8ea80850b743a51a1320104455445ee650a146b7288ff763fc30ea20200659eb WatchSource:0}: Error finding container 8ea80850b743a51a1320104455445ee650a146b7288ff763fc30ea20200659eb: Status 404 returned error can't find the container with id 8ea80850b743a51a1320104455445ee650a146b7288ff763fc30ea20200659eb
Jul 14 19:49:05.270771 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:05.270728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b54df4f4d-5xczm" event={"ID":"7d5a6d6d-4c30-4488-aed6-1a5ad257602c","Type":"ContainerStarted","Data":"eff21bb4c4e0acffb1bf6c0cdc208b7b47df37004d84bf236220b2fe6e382ef4"}
Jul 14 19:49:05.270977 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:05.270795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b54df4f4d-5xczm" event={"ID":"7d5a6d6d-4c30-4488-aed6-1a5ad257602c","Type":"ContainerStarted","Data":"8ea80850b743a51a1320104455445ee650a146b7288ff763fc30ea20200659eb"}
Jul 14 19:49:05.289565 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:05.289508    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b54df4f4d-5xczm" podStartSLOduration=1.2894882189999999 podStartE2EDuration="1.289488219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:49:05.288108263 +0000 UTC m=+643.599807720" watchObservedRunningTime="2026-07-14 19:49:05.289488219 +0000 UTC m=+643.601187675"
Jul 14 19:49:12.201192 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.201148    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:49:12.205293 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.205266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.207782 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.207760    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j6wm7\""
Jul 14 19:49:12.207923 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.207760    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 19:49:12.214898 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.214862    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:49:12.239138 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.239099    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:49:12.301592 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.301552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/709fb926-6a75-49ca-ba6f-38fca2f82557-config-file\") pod \"limitador-limitador-54d6d988d5-cj2mn\" (UID: \"709fb926-6a75-49ca-ba6f-38fca2f82557\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.301780 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.301607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkf7n\" (UniqueName: \"kubernetes.io/projected/709fb926-6a75-49ca-ba6f-38fca2f82557-kube-api-access-mkf7n\") pod \"limitador-limitador-54d6d988d5-cj2mn\" (UID: \"709fb926-6a75-49ca-ba6f-38fca2f82557\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.403349 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.403306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/709fb926-6a75-49ca-ba6f-38fca2f82557-config-file\") pod \"limitador-limitador-54d6d988d5-cj2mn\" (UID: \"709fb926-6a75-49ca-ba6f-38fca2f82557\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.403588 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.403387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkf7n\" (UniqueName: \"kubernetes.io/projected/709fb926-6a75-49ca-ba6f-38fca2f82557-kube-api-access-mkf7n\") pod \"limitador-limitador-54d6d988d5-cj2mn\" (UID: \"709fb926-6a75-49ca-ba6f-38fca2f82557\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.404104 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.404079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/709fb926-6a75-49ca-ba6f-38fca2f82557-config-file\") pod \"limitador-limitador-54d6d988d5-cj2mn\" (UID: \"709fb926-6a75-49ca-ba6f-38fca2f82557\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.411909 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.411878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkf7n\" (UniqueName: \"kubernetes.io/projected/709fb926-6a75-49ca-ba6f-38fca2f82557-kube-api-access-mkf7n\") pod \"limitador-limitador-54d6d988d5-cj2mn\" (UID: \"709fb926-6a75-49ca-ba6f-38fca2f82557\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.518035 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.517934    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:12.671673 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:12.671640    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:49:12.673202 ip-10-0-130-104 kubenswrapper[2581]: W0714 19:49:12.673174    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709fb926_6a75_49ca_ba6f_38fca2f82557.slice/crio-4ec22bf7819919f5679af3c8bd3f66bac76b61abe78cb1e8b99de9dd3fae7172 WatchSource:0}: Error finding container 4ec22bf7819919f5679af3c8bd3f66bac76b61abe78cb1e8b99de9dd3fae7172: Status 404 returned error can't find the container with id 4ec22bf7819919f5679af3c8bd3f66bac76b61abe78cb1e8b99de9dd3fae7172
Jul 14 19:49:13.318539 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:13.318501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn" event={"ID":"709fb926-6a75-49ca-ba6f-38fca2f82557","Type":"ContainerStarted","Data":"4ec22bf7819919f5679af3c8bd3f66bac76b61abe78cb1e8b99de9dd3fae7172"}
Jul 14 19:49:14.806521 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:14.806480    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:14.807019 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:14.806557    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:14.812728 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:14.812701    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:15.339170 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:15.339140    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b54df4f4d-5xczm"
Jul 14 19:49:15.390449 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:15.390363    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5594c478-qklfc"]
Jul 14 19:49:16.341256 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:16.341205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn" event={"ID":"709fb926-6a75-49ca-ba6f-38fca2f82557","Type":"ContainerStarted","Data":"da84a45d6ccb4e300dd112496d83cf0ca0f934753a7771db723d762af4646d78"}
Jul 14 19:49:16.341769 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:16.341386    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:16.360001 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:16.359945    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn" podStartSLOduration=1.159638266 podStartE2EDuration="4.359926756s" totalImagesPullingTime="3.20028849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 19:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 19:49:16.35806128 +0000 UTC m=+654.669760737" watchObservedRunningTime="2026-07-14 19:49:16.359926756 +0000 UTC m=+654.671626211"
Jul 14 19:49:27.348118 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:27.348087    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"
Jul 14 19:49:40.414662 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.414590    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f5594c478-qklfc" podUID="ab6ebd30-9871-462f-a06c-0bd3b1e25452" containerName="console" containerID="cri-o://c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe" gracePeriod=15
Jul 14 19:49:40.685392 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.685366    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:49:40.779683 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.779639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-oauth-config\") pod \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") "
Jul 14 19:49:40.779683 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.779681    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-config\") pod \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") "
Jul 14 19:49:40.779956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.779764    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps8sc\" (UniqueName: \"kubernetes.io/projected/ab6ebd30-9871-462f-a06c-0bd3b1e25452-kube-api-access-ps8sc\") pod \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") "
Jul 14 19:49:40.779956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.779795    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-serving-cert\") pod \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") "
Jul 14 19:49:40.779956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.779837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-oauth-serving-cert\") pod \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") "
Jul 14 19:49:40.779956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.779879    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-service-ca\") pod \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") "
Jul 14 19:49:40.779956 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.779916    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-trusted-ca-bundle\") pod \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\" (UID: \"ab6ebd30-9871-462f-a06c-0bd3b1e25452\") "
Jul 14 19:49:40.780373 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.780341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-config" (OuterVolumeSpecName: "console-config") pod "ab6ebd30-9871-462f-a06c-0bd3b1e25452" (UID: "ab6ebd30-9871-462f-a06c-0bd3b1e25452"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:49:40.780548 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.780356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ab6ebd30-9871-462f-a06c-0bd3b1e25452" (UID: "ab6ebd30-9871-462f-a06c-0bd3b1e25452"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:49:40.780548 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.780471    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-service-ca" (OuterVolumeSpecName: "service-ca") pod "ab6ebd30-9871-462f-a06c-0bd3b1e25452" (UID: "ab6ebd30-9871-462f-a06c-0bd3b1e25452"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:49:40.780701 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.780564    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ab6ebd30-9871-462f-a06c-0bd3b1e25452" (UID: "ab6ebd30-9871-462f-a06c-0bd3b1e25452"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 19:49:40.782048 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.782020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ab6ebd30-9871-462f-a06c-0bd3b1e25452" (UID: "ab6ebd30-9871-462f-a06c-0bd3b1e25452"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:49:40.782137 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.782071    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab6ebd30-9871-462f-a06c-0bd3b1e25452-kube-api-access-ps8sc" (OuterVolumeSpecName: "kube-api-access-ps8sc") pod "ab6ebd30-9871-462f-a06c-0bd3b1e25452" (UID: "ab6ebd30-9871-462f-a06c-0bd3b1e25452"). InnerVolumeSpecName "kube-api-access-ps8sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:49:40.782182 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.782168    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ab6ebd30-9871-462f-a06c-0bd3b1e25452" (UID: "ab6ebd30-9871-462f-a06c-0bd3b1e25452"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 19:49:40.881742 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.881695    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps8sc\" (UniqueName: \"kubernetes.io/projected/ab6ebd30-9871-462f-a06c-0bd3b1e25452-kube-api-access-ps8sc\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:49:40.881742 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.881735    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-serving-cert\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:49:40.881742 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.881747    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-oauth-serving-cert\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:49:40.882009 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.881756    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:49:40.882009 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.881766    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-trusted-ca-bundle\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:49:40.882009 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.881774    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-oauth-config\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:49:40.882009 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:40.881784    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab6ebd30-9871-462f-a06c-0bd3b1e25452-console-config\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:49:41.488218 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.488180    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab6ebd30-9871-462f-a06c-0bd3b1e25452" containerID="c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe" exitCode=0
Jul 14 19:49:41.488646 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.488226    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5594c478-qklfc"
Jul 14 19:49:41.488646 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.488264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5594c478-qklfc" event={"ID":"ab6ebd30-9871-462f-a06c-0bd3b1e25452","Type":"ContainerDied","Data":"c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe"}
Jul 14 19:49:41.488646 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.488325    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5594c478-qklfc" event={"ID":"ab6ebd30-9871-462f-a06c-0bd3b1e25452","Type":"ContainerDied","Data":"bef129a68a09ab6606aebbd18217aa41ffbd27fc6edfb97ef46ba965da33b15e"}
Jul 14 19:49:41.488646 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.488348    2581 scope.go:117] "RemoveContainer" containerID="c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe"
Jul 14 19:49:41.497946 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.497925    2581 scope.go:117] "RemoveContainer" containerID="c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe"
Jul 14 19:49:41.498247 ip-10-0-130-104 kubenswrapper[2581]: E0714 19:49:41.498226    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe\": container with ID starting with c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe not found: ID does not exist" containerID="c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe"
Jul 14 19:49:41.498318 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.498256    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe"} err="failed to get container status \"c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe\": rpc error: code = NotFound desc = could not find container \"c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe\": container with ID starting with c2d6422e664e49b667e8e71ec51e9ad76d0abed15f54f8071d81835a6c8d4efe not found: ID does not exist"
Jul 14 19:49:41.508865 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.508838    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5594c478-qklfc"]
Jul 14 19:49:41.513280 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:41.513252    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f5594c478-qklfc"]
Jul 14 19:49:43.339920 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:49:43.339880    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab6ebd30-9871-462f-a06c-0bd3b1e25452" path="/var/lib/kubelet/pods/ab6ebd30-9871-462f-a06c-0bd3b1e25452/volumes"
Jul 14 19:50:27.978127 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:27.978091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:50:36.828091 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:36.828057    2581 generic.go:356] "Generic (PLEG): container finished" podID="18ad6f09-f21e-4cb1-b810-a03ca4a4b180" containerID="05336926d1509d97b7e8b8160ec2aca5fc21ea602e0fcc440bad72425d001177" exitCode=0
Jul 14 19:50:36.828565 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:36.828139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" event={"ID":"18ad6f09-f21e-4cb1-b810-a03ca4a4b180","Type":"ContainerDied","Data":"05336926d1509d97b7e8b8160ec2aca5fc21ea602e0fcc440bad72425d001177"}
Jul 14 19:50:36.828611 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:36.828566    2581 scope.go:117] "RemoveContainer" containerID="05336926d1509d97b7e8b8160ec2aca5fc21ea602e0fcc440bad72425d001177"
Jul 14 19:50:36.829597 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:36.829565    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:50:36.829705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:36.829636    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:50:36.829705 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:36.829689    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 19:50:37.835492 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:37.835453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" event={"ID":"18ad6f09-f21e-4cb1-b810-a03ca4a4b180","Type":"ContainerStarted","Data":"3ef8df5ea3145f238fc3ac35603100897b9f92990b01ecc3d335264dd5edf11b"}
Jul 14 19:50:38.528699 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:38.528656    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:50:38.528954 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:38.528929    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thcwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 14 19:50:38.529001 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:38.528980    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" podUID="18ad6f09-f21e-4cb1-b810-a03ca4a4b180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 14 19:50:39.529097 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:39.529060    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thcwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 14 19:50:39.529549 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:39.529139    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" podUID="18ad6f09-f21e-4cb1-b810-a03ca4a4b180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 14 19:50:40.529501 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:40.529451    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thcwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 14 19:50:40.530027 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:40.529540    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" podUID="18ad6f09-f21e-4cb1-b810-a03ca4a4b180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 14 19:50:41.529914 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:41.529858    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thcwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 14 19:50:41.530347 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:41.529935    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq" podUID="18ad6f09-f21e-4cb1-b810-a03ca4a4b180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 14 19:50:42.532879 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:42.532844    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:50:42.533288 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:42.533122    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:50:42.534078 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:42.534060    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thcwq"
Jul 14 19:50:49.772594 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:49.772548    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:50:56.069213 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:50:56.069170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:51:04.346374 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:51:04.346294    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:51:17.539181 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:51:17.539138    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:51:30.832742 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:51:30.832697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:51:44.231008 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:51:44.230927    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:53:06.538147 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:53:06.538103    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:53:16.232816 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:53:16.232725    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:53:23.248674 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:53:23.248647    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:53:23.250766 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:53:23.250737    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:53:27.634282 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:53:27.634237    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:53:35.627924 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:53:35.627883    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:53:47.626923 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:53:47.626880    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:54:15.932118 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:54:15.932084    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:54:26.864530 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:54:26.864488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:55:13.157657 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:55:13.157599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:55:22.589291 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:55:22.589256    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:55:35.935058 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:55:35.935016    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:55:42.760788 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:55:42.760740    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:55:54.063870 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:55:54.063826    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:56:04.447110 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:56:04.447067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:56:17.465807 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:56:17.465705    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:56:26.533849 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:56:26.533805    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:56:54.643609 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:56:54.643562    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:57:06.831917 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:57:06.831874    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:57:30.432756 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:57:30.432717    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:57:41.829696 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:57:41.829654    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:57:56.730432 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:57:56.730336    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:58:09.131069 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:58:09.131029    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:58:23.287511 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:58:23.287483    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:58:23.292178 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:58:23.292151    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 19:58:25.630643 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:58:25.630605    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:58:35.846970 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:58:35.846925    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 19:59:02.599499 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:02.599460    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"]
Jul 14 19:59:02.600110 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:02.599677    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz" podUID="80681db0-5c90-45ae-a162-ddc8a14f833e" containerName="manager" containerID="cri-o://0b0297c32cf62df1945cdaa95352372090d6b61db83ca2a2181b40c32f940a6d" gracePeriod=10
Jul 14 19:59:05.886211 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:05.886172    2581 generic.go:356] "Generic (PLEG): container finished" podID="80681db0-5c90-45ae-a162-ddc8a14f833e" containerID="0b0297c32cf62df1945cdaa95352372090d6b61db83ca2a2181b40c32f940a6d" exitCode=0
Jul 14 19:59:05.886628 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:05.886233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz" event={"ID":"80681db0-5c90-45ae-a162-ddc8a14f833e","Type":"ContainerDied","Data":"0b0297c32cf62df1945cdaa95352372090d6b61db83ca2a2181b40c32f940a6d"}
Jul 14 19:59:05.968076 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:05.968042    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:59:05.986172 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:05.986138    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x22qq\" (UniqueName: \"kubernetes.io/projected/80681db0-5c90-45ae-a162-ddc8a14f833e-kube-api-access-x22qq\") pod \"80681db0-5c90-45ae-a162-ddc8a14f833e\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") "
Jul 14 19:59:05.986328 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:05.986275    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/80681db0-5c90-45ae-a162-ddc8a14f833e-extensions-socket-volume\") pod \"80681db0-5c90-45ae-a162-ddc8a14f833e\" (UID: \"80681db0-5c90-45ae-a162-ddc8a14f833e\") "
Jul 14 19:59:05.986760 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:05.986708    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80681db0-5c90-45ae-a162-ddc8a14f833e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "80681db0-5c90-45ae-a162-ddc8a14f833e" (UID: "80681db0-5c90-45ae-a162-ddc8a14f833e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 19:59:05.988997 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:05.988962    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80681db0-5c90-45ae-a162-ddc8a14f833e-kube-api-access-x22qq" (OuterVolumeSpecName: "kube-api-access-x22qq") pod "80681db0-5c90-45ae-a162-ddc8a14f833e" (UID: "80681db0-5c90-45ae-a162-ddc8a14f833e"). InnerVolumeSpecName "kube-api-access-x22qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 19:59:06.086989 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:06.086871    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/80681db0-5c90-45ae-a162-ddc8a14f833e-extensions-socket-volume\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:59:06.086989 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:06.086912    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x22qq\" (UniqueName: \"kubernetes.io/projected/80681db0-5c90-45ae-a162-ddc8a14f833e-kube-api-access-x22qq\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 19:59:06.894257 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:06.894221    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"
Jul 14 19:59:06.894749 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:06.894220    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz" event={"ID":"80681db0-5c90-45ae-a162-ddc8a14f833e","Type":"ContainerDied","Data":"f73b3ee89f6dcbefc66174ad773bc13dd3e0656996e5c94d70eff9c10b14915e"}
Jul 14 19:59:06.894749 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:06.894361    2581 scope.go:117] "RemoveContainer" containerID="0b0297c32cf62df1945cdaa95352372090d6b61db83ca2a2181b40c32f940a6d"
Jul 14 19:59:06.922621 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:06.922588    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"]
Jul 14 19:59:06.924945 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:06.924921    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xzhpz"]
Jul 14 19:59:07.339147 ip-10-0-130-104 kubenswrapper[2581]: I0714 19:59:07.339102    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80681db0-5c90-45ae-a162-ddc8a14f833e" path="/var/lib/kubelet/pods/80681db0-5c90-45ae-a162-ddc8a14f833e/volumes"
Jul 14 20:00:00.242391 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.242354    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666"]
Jul 14 20:00:00.242958 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.242864    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80681db0-5c90-45ae-a162-ddc8a14f833e" containerName="manager"
Jul 14 20:00:00.242958 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.242877    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="80681db0-5c90-45ae-a162-ddc8a14f833e" containerName="manager"
Jul 14 20:00:00.242958 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.242890    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab6ebd30-9871-462f-a06c-0bd3b1e25452" containerName="console"
Jul 14 20:00:00.242958 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.242896    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6ebd30-9871-462f-a06c-0bd3b1e25452" containerName="console"
Jul 14 20:00:00.242958 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.242955    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab6ebd30-9871-462f-a06c-0bd3b1e25452" containerName="console"
Jul 14 20:00:00.243150 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.242968    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="80681db0-5c90-45ae-a162-ddc8a14f833e" containerName="manager"
Jul 14 20:00:00.246329 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.246312    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666"
Jul 14 20:00:00.249897 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.249872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:00:00.256424 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.256391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666"
Jul 14 20:00:00.264103 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.264074    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666"]
Jul 14 20:00:00.419913 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.419881    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666"]
Jul 14 20:00:00.422664 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:00:00.422627    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod747d85e1_bacb_4702_b509_c3aa03de87dc.slice/crio-7da458b511236696497bcfdffd690494549f4fac81381b410327167cd3ec0d71 WatchSource:0}: Error finding container 7da458b511236696497bcfdffd690494549f4fac81381b410327167cd3ec0d71: Status 404 returned error can't find the container with id 7da458b511236696497bcfdffd690494549f4fac81381b410327167cd3ec0d71
Jul 14 20:00:00.424745 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:00.424723    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 20:00:01.211087 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:01.211045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666" event={"ID":"747d85e1-bacb-4702-b509-c3aa03de87dc","Type":"ContainerStarted","Data":"7da458b511236696497bcfdffd690494549f4fac81381b410327167cd3ec0d71"}
Jul 14 20:00:03.786065 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.786017    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"]
Jul 14 20:00:03.791279 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.791249    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:03.794455 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.794423    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8tl8d\""
Jul 14 20:00:03.802321 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.802290    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"]
Jul 14 20:00:03.832650 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.832601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqmtv\" (UniqueName: \"kubernetes.io/projected/62502f27-1433-4130-bc31-f718ab3ceab2-kube-api-access-dqmtv\") pod \"kuadrant-operator-controller-manager-86878c769f-xb5rh\" (UID: \"62502f27-1433-4130-bc31-f718ab3ceab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:03.832861 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.832669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62502f27-1433-4130-bc31-f718ab3ceab2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xb5rh\" (UID: \"62502f27-1433-4130-bc31-f718ab3ceab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:03.934172 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.934126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqmtv\" (UniqueName: \"kubernetes.io/projected/62502f27-1433-4130-bc31-f718ab3ceab2-kube-api-access-dqmtv\") pod \"kuadrant-operator-controller-manager-86878c769f-xb5rh\" (UID: \"62502f27-1433-4130-bc31-f718ab3ceab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:03.934371 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.934192    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62502f27-1433-4130-bc31-f718ab3ceab2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xb5rh\" (UID: \"62502f27-1433-4130-bc31-f718ab3ceab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:03.934627 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.934604    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62502f27-1433-4130-bc31-f718ab3ceab2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xb5rh\" (UID: \"62502f27-1433-4130-bc31-f718ab3ceab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:03.943763 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:03.943734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqmtv\" (UniqueName: \"kubernetes.io/projected/62502f27-1433-4130-bc31-f718ab3ceab2-kube-api-access-dqmtv\") pod \"kuadrant-operator-controller-manager-86878c769f-xb5rh\" (UID: \"62502f27-1433-4130-bc31-f718ab3ceab2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:04.105694 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:04.105575    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:04.265249 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:04.265219    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"]
Jul 14 20:00:04.268104 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:00:04.268072    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62502f27_1433_4130_bc31_f718ab3ceab2.slice/crio-7f06c057f4de85d2e82d30f9ff75cf7ec2df8d36827ff9fb3024dc0123f6b4bf WatchSource:0}: Error finding container 7f06c057f4de85d2e82d30f9ff75cf7ec2df8d36827ff9fb3024dc0123f6b4bf: Status 404 returned error can't find the container with id 7f06c057f4de85d2e82d30f9ff75cf7ec2df8d36827ff9fb3024dc0123f6b4bf
Jul 14 20:00:05.239817 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:05.239778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh" event={"ID":"62502f27-1433-4130-bc31-f718ab3ceab2","Type":"ContainerStarted","Data":"a3918c3deff4a6e2c670a829cc1e0c8d62bc1b1f471d95cebe8e1b9374503833"}
Jul 14 20:00:05.240270 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:05.239831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh" event={"ID":"62502f27-1433-4130-bc31-f718ab3ceab2","Type":"ContainerStarted","Data":"7f06c057f4de85d2e82d30f9ff75cf7ec2df8d36827ff9fb3024dc0123f6b4bf"}
Jul 14 20:00:05.240270 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:05.239909    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:05.263992 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:05.263930    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh" podStartSLOduration=2.263907345 podStartE2EDuration="2.263907345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:00:05.261284509 +0000 UTC m=+1303.572983966" watchObservedRunningTime="2026-07-14 20:00:05.263907345 +0000 UTC m=+1303.575606801"
Jul 14 20:00:16.248538 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:16.248499    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xb5rh"
Jul 14 20:00:20.331157 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:20.331117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666" event={"ID":"747d85e1-bacb-4702-b509-c3aa03de87dc","Type":"ContainerStarted","Data":"d22df6bf0f5f528a6af9c0c38a0cb8c01a145f958977f9af8bb351a8ab903ddf"}
Jul 14 20:00:20.352926 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:20.352867    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666" podStartSLOduration=1.273443013 podStartE2EDuration="20.352848498s" totalImagesPullingTime="19.079405485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:00:20.3511599 +0000 UTC m=+1318.662859356" watchObservedRunningTime="2026-07-14 20:00:20.352848498 +0000 UTC m=+1318.664547954"
Jul 14 20:00:21.338811 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:21.338775    2581 generic.go:356] "Generic (PLEG): container finished" podID="747d85e1-bacb-4702-b509-c3aa03de87dc" containerID="d22df6bf0f5f528a6af9c0c38a0cb8c01a145f958977f9af8bb351a8ab903ddf" exitCode=0
Jul 14 20:00:21.339290 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:21.338857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666" event={"ID":"747d85e1-bacb-4702-b509-c3aa03de87dc","Type":"ContainerDied","Data":"d22df6bf0f5f528a6af9c0c38a0cb8c01a145f958977f9af8bb351a8ab903ddf"}
Jul 14 20:00:22.496247 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:22.496219    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666"
Jul 14 20:00:23.353866 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:23.353835    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666" event={"ID":"747d85e1-bacb-4702-b509-c3aa03de87dc","Type":"ContainerDied","Data":"7da458b511236696497bcfdffd690494549f4fac81381b410327167cd3ec0d71"}
Jul 14 20:00:23.354005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:23.353895    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7da458b511236696497bcfdffd690494549f4fac81381b410327167cd3ec0d71"
Jul 14 20:00:23.354005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:23.353846    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734320-sr666"
Jul 14 20:00:25.168281 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:25.168246    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:00:34.729856 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:34.729819    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:00:47.732900 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:00:47.732814    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:01:00.038750 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:01:00.038711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:01:09.745911 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:01:09.745860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:01:20.939185 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:01:20.939142    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:01:29.230544 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:01:29.230500    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:01:40.530709 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:01:40.530671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:01:52.432892 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:01:52.432849    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:02:01.533575 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:02:01.533530    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:02:15.332168 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:02:15.332075    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:02:36.049572 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:02:36.049534    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:02:43.933191 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:02:43.933148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:02:54.048732 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:02:54.048683    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:03:03.929221 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:03.929175    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:03:13.552726 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:13.552686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:03:23.325854 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:23.325827    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:03:23.332838 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:23.332804    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:03:25.930424 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:25.930371    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:03:37.731372 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:37.731331    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:03:47.829794 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:47.829704    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:03:56.330518 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:03:56.330472    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:04:05.034886 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:04:05.034842    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:04:14.831177 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:04:14.831131    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:04:29.425771 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:04:29.425729    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:04:38.640472 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:04:38.640426    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:04:48.539502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:04:48.539457    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:04:58.432198 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:04:58.432159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:05:10.924186 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:05:10.924146    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:05:21.536413 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:05:21.536309    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:05:34.034359 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:05:34.034319    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:05:47.537621 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:05:47.537585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:06:03.735709 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:03.735675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:06:16.332788 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:16.332742    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:06:25.827202 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:25.827154    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:06:35.368369 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:35.368324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:06:36.692302 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.692265    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"]
Jul 14 20:06:36.692937 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.692916    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747d85e1-bacb-4702-b509-c3aa03de87dc" containerName="cleanup"
Jul 14 20:06:36.693030 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.692940    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="747d85e1-bacb-4702-b509-c3aa03de87dc" containerName="cleanup"
Jul 14 20:06:36.693084 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.693039    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="747d85e1-bacb-4702-b509-c3aa03de87dc" containerName="cleanup"
Jul 14 20:06:36.696381 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.696355    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.699057 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.699032    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ffd1ab16-openshift-default-dockercfg-m4wc8\""
Jul 14 20:06:36.723589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.723551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"]
Jul 14 20:06:36.811336 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-socket\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-data\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-certs\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811520    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-podinfo\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926646fb-eaca-4fcf-ad59-3f82e89eb851-istiod-ca-cert\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811844 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-envoy\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811844 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgtmx\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-kube-api-access-fgtmx\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811844 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-credential-socket\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.811844 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.811698    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-token\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912522 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-data\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-certs\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-podinfo\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926646fb-eaca-4fcf-ad59-3f82e89eb851-istiod-ca-cert\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-envoy\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgtmx\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-kube-api-access-fgtmx\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-credential-socket\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.912725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-token\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.913102 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-data\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.913102 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.912940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-certs\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.913203 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.913163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-socket\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.913305 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.913280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926646fb-eaca-4fcf-ad59-3f82e89eb851-istiod-ca-cert\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.913539 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.913518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-socket\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.913616 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.913592    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-credential-socket\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.915116 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.915091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-envoy\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.915484 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.915468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-podinfo\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.921695 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.921671    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-token\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:36.921994 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:36.921965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgtmx\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-kube-api-access-fgtmx\") pod \"e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") " pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:37.008255 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.008208    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:37.165339 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.165306    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"]
Jul 14 20:06:37.166816 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:06:37.166784    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926646fb_eaca_4fcf_ad59_3f82e89eb851.slice/crio-b3c8efcc788c97c7a09c70d51775ff2693958149fe5425e6f448e01474b51849 WatchSource:0}: Error finding container b3c8efcc788c97c7a09c70d51775ff2693958149fe5425e6f448e01474b51849: Status 404 returned error can't find the container with id b3c8efcc788c97c7a09c70d51775ff2693958149fe5425e6f448e01474b51849
Jul 14 20:06:37.170032 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.169988    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 20:06:37.170127 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.170084    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 20:06:37.170176 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.170127    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 20:06:37.628327 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.628288    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" event={"ID":"926646fb-eaca-4fcf-ad59-3f82e89eb851","Type":"ContainerStarted","Data":"e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72"}
Jul 14 20:06:37.628566 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.628347    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" event={"ID":"926646fb-eaca-4fcf-ad59-3f82e89eb851","Type":"ContainerStarted","Data":"b3c8efcc788c97c7a09c70d51775ff2693958149fe5425e6f448e01474b51849"}
Jul 14 20:06:37.649876 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:37.649809    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podStartSLOduration=1.649786145 podStartE2EDuration="1.649786145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:06:37.647475455 +0000 UTC m=+1695.959174911" watchObservedRunningTime="2026-07-14 20:06:37.649786145 +0000 UTC m=+1695.961485602"
Jul 14 20:06:38.008716 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:38.008673    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:38.014141 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:38.014095    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:38.634673 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:38.634640    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:38.635820 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:38.635799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:42.977027 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:42.976985    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"]
Jul 14 20:06:42.981006 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:42.980977    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"
Jul 14 20:06:42.983453 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:42.983425    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:06:42.984506 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:42.984475    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:06:42.986890 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:42.986861    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"]
Jul 14 20:06:43.074814 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:43.074768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e838137-a2d2-4d42-8549-1ab63ea30ca2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw\" (UID: \"0e838137-a2d2-4d42-8549-1ab63ea30ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"
Jul 14 20:06:43.175771 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:43.175723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e838137-a2d2-4d42-8549-1ab63ea30ca2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw\" (UID: \"0e838137-a2d2-4d42-8549-1ab63ea30ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"
Jul 14 20:06:43.176449 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:43.176392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e838137-a2d2-4d42-8549-1ab63ea30ca2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw\" (UID: \"0e838137-a2d2-4d42-8549-1ab63ea30ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"
Jul 14 20:06:43.295080 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:43.294976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"
Jul 14 20:06:43.440998 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:43.440970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"]
Jul 14 20:06:43.443856 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:06:43.443825    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e838137_a2d2_4d42_8549_1ab63ea30ca2.slice/crio-bd7784b65ad4f0133fcbc84f181adff5c790ced7bc2e29eb5f0131e8b2083094 WatchSource:0}: Error finding container bd7784b65ad4f0133fcbc84f181adff5c790ced7bc2e29eb5f0131e8b2083094: Status 404 returned error can't find the container with id bd7784b65ad4f0133fcbc84f181adff5c790ced7bc2e29eb5f0131e8b2083094
Jul 14 20:06:43.678230 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:43.678048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw" event={"ID":"0e838137-a2d2-4d42-8549-1ab63ea30ca2","Type":"ContainerStarted","Data":"bd7784b65ad4f0133fcbc84f181adff5c790ced7bc2e29eb5f0131e8b2083094"}
Jul 14 20:06:44.685773 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:44.685740    2581 generic.go:356] "Generic (PLEG): container finished" podID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerID="f9988bb71ac51dc00c98faf7b20c94141c0450e741db3b97b6e9cda4326c2a5e" exitCode=7
Jul 14 20:06:44.686180 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:44.685818    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw" event={"ID":"0e838137-a2d2-4d42-8549-1ab63ea30ca2","Type":"ContainerDied","Data":"f9988bb71ac51dc00c98faf7b20c94141c0450e741db3b97b6e9cda4326c2a5e"}
Jul 14 20:06:44.686180 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:44.685984    2581 scope.go:117] "RemoveContainer" containerID="f9988bb71ac51dc00c98faf7b20c94141c0450e741db3b97b6e9cda4326c2a5e"
Jul 14 20:06:45.695155 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:45.695056    2581 generic.go:356] "Generic (PLEG): container finished" podID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerID="40fa4316b70e9fa1c3a11b74944adc20a295eb86e70bc88483136d62c76e6a97" exitCode=7
Jul 14 20:06:45.695155 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:45.695141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw" event={"ID":"0e838137-a2d2-4d42-8549-1ab63ea30ca2","Type":"ContainerDied","Data":"40fa4316b70e9fa1c3a11b74944adc20a295eb86e70bc88483136d62c76e6a97"}
Jul 14 20:06:45.695693 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:45.695205    2581 scope.go:117] "RemoveContainer" containerID="f9988bb71ac51dc00c98faf7b20c94141c0450e741db3b97b6e9cda4326c2a5e"
Jul 14 20:06:45.695693 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:45.695483    2581 scope.go:117] "RemoveContainer" containerID="40fa4316b70e9fa1c3a11b74944adc20a295eb86e70bc88483136d62c76e6a97"
Jul 14 20:06:45.695783 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:06:45.695736    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw_odh-ai-gateway-infra(0e838137-a2d2-4d42-8549-1ab63ea30ca2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2"
Jul 14 20:06:46.703805 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:46.703772    2581 scope.go:117] "RemoveContainer" containerID="40fa4316b70e9fa1c3a11b74944adc20a295eb86e70bc88483136d62c76e6a97"
Jul 14 20:06:46.704190 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:06:46.703966    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw_odh-ai-gateway-infra(0e838137-a2d2-4d42-8549-1ab63ea30ca2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2"
Jul 14 20:06:46.756280 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:46.756235    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:06:54.676582 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:54.676540    2581 patch_prober.go:28] interesting pod/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 14 20:06:54.677019 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:54.676607    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 14 20:06:55.763637 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:55.763595    2581 generic.go:356] "Generic (PLEG): container finished" podID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerID="e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72" exitCode=0
Jul 14 20:06:55.764082 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:55.763671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" event={"ID":"926646fb-eaca-4fcf-ad59-3f82e89eb851","Type":"ContainerDied","Data":"e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72"}
Jul 14 20:06:55.764191 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:55.764171    2581 scope.go:117] "RemoveContainer" containerID="e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72"
Jul 14 20:06:55.765375 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:55.765342    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 20:06:55.765455 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:55.765443    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 20:06:55.765502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:55.765484    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 14 20:06:56.773032 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:56.772997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" event={"ID":"926646fb-eaca-4fcf-ad59-3f82e89eb851","Type":"ContainerStarted","Data":"773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c"}
Jul 14 20:06:57.008467 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:57.008420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:06:57.008660 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:57.008638    2581 patch_prober.go:28] interesting pod/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 14 20:06:57.008723 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:57.008686    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 14 20:06:58.008949 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:58.008908    2581 patch_prober.go:28] interesting pod/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 14 20:06:58.009370 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:58.008964    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 14 20:06:59.009551 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:59.009506    2581 patch_prober.go:28] interesting pod/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 14 20:06:59.009984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:06:59.009569    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 14 20:07:00.009623 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:00.009582    2581 patch_prober.go:28] interesting pod/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 14 20:07:00.010070 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:00.009644    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 14 20:07:00.336567 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:00.336459    2581 scope.go:117] "RemoveContainer" containerID="40fa4316b70e9fa1c3a11b74944adc20a295eb86e70bc88483136d62c76e6a97"
Jul 14 20:07:01.009425 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:01.009358    2581 patch_prober.go:28] interesting pod/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 14 20:07:01.009613 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:01.009449    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 14 20:07:01.812227 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:01.812138    2581 generic.go:356] "Generic (PLEG): container finished" podID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerID="f6a24f8f78c7658c40c8692fe73c984d7c41c98b1c9c93b00ab38ff2f5d89e33" exitCode=0
Jul 14 20:07:01.812227 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:01.812205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw" event={"ID":"0e838137-a2d2-4d42-8549-1ab63ea30ca2","Type":"ContainerDied","Data":"f6a24f8f78c7658c40c8692fe73c984d7c41c98b1c9c93b00ab38ff2f5d89e33"}
Jul 14 20:07:01.812732 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:01.812269    2581 scope.go:117] "RemoveContainer" containerID="40fa4316b70e9fa1c3a11b74944adc20a295eb86e70bc88483136d62c76e6a97"
Jul 14 20:07:02.012929 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:02.012900    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:07:02.013315 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:02.013286    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:07:02.014163 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:02.014143    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:07:02.840801 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:02.840757    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"]
Jul 14 20:07:02.975983 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:02.975953    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"
Jul 14 20:07:03.076077 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.075968    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e838137-a2d2-4d42-8549-1ab63ea30ca2-maas-api-service-ca\") pod \"0e838137-a2d2-4d42-8549-1ab63ea30ca2\" (UID: \"0e838137-a2d2-4d42-8549-1ab63ea30ca2\") "
Jul 14 20:07:03.076377 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.076351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e838137-a2d2-4d42-8549-1ab63ea30ca2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e838137-a2d2-4d42-8549-1ab63ea30ca2" (UID: "0e838137-a2d2-4d42-8549-1ab63ea30ca2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:07:03.176978 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.176931    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e838137-a2d2-4d42-8549-1ab63ea30ca2-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:07:03.827918 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.827887    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"
Jul 14 20:07:03.828108 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.827922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw" event={"ID":"0e838137-a2d2-4d42-8549-1ab63ea30ca2","Type":"ContainerDied","Data":"bd7784b65ad4f0133fcbc84f181adff5c790ced7bc2e29eb5f0131e8b2083094"}
Jul 14 20:07:03.828108 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.827971    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd7784b65ad4f0133fcbc84f181adff5c790ced7bc2e29eb5f0131e8b2083094"
Jul 14 20:07:03.866137 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.866093    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"]
Jul 14 20:07:03.869867 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:03.869835    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-4kgpw"]
Jul 14 20:07:05.339461 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:05.339424    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" path="/var/lib/kubelet/pods/0e838137-a2d2-4d42-8549-1ab63ea30ca2/volumes"
Jul 14 20:07:43.039563 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.039524    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"]
Jul 14 20:07:43.040076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.039976    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.039988    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.039997    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.040003    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.040008    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.040014    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040275 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.040090    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040275 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.040097    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.040275 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.040104    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e838137-a2d2-4d42-8549-1ab63ea30ca2" containerName="revoke-keys"
Jul 14 20:07:43.042246 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.042227    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"
Jul 14 20:07:43.045108 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.045084    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:07:43.045231 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.045171    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:07:43.054285 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.054248    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"]
Jul 14 20:07:43.161691 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.161651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb4aa24-aeed-4145-bd9b-62da3c95993c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc\" (UID: \"dbb4aa24-aeed-4145-bd9b-62da3c95993c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"
Jul 14 20:07:43.262207 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.262162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb4aa24-aeed-4145-bd9b-62da3c95993c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc\" (UID: \"dbb4aa24-aeed-4145-bd9b-62da3c95993c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"
Jul 14 20:07:43.262894 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.262869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb4aa24-aeed-4145-bd9b-62da3c95993c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc\" (UID: \"dbb4aa24-aeed-4145-bd9b-62da3c95993c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"
Jul 14 20:07:43.354999 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.354906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"
Jul 14 20:07:43.518583 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:43.518556    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"]
Jul 14 20:07:43.520909 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:07:43.520881    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb4aa24_aeed_4145_bd9b_62da3c95993c.slice/crio-3c10fa88ada3dc0743512a2f8d397d4d9278729a998476a8046ed4a99308053c WatchSource:0}: Error finding container 3c10fa88ada3dc0743512a2f8d397d4d9278729a998476a8046ed4a99308053c: Status 404 returned error can't find the container with id 3c10fa88ada3dc0743512a2f8d397d4d9278729a998476a8046ed4a99308053c
Jul 14 20:07:44.074080 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:44.074043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc" event={"ID":"dbb4aa24-aeed-4145-bd9b-62da3c95993c","Type":"ContainerStarted","Data":"b3a0b84677950e97a659ae7326c15d3ac296d9a0b502b8262ed3e4423e8ca514"}
Jul 14 20:07:44.074502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:44.074105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc" event={"ID":"dbb4aa24-aeed-4145-bd9b-62da3c95993c","Type":"ContainerStarted","Data":"3c10fa88ada3dc0743512a2f8d397d4d9278729a998476a8046ed4a99308053c"}
Jul 14 20:07:44.093568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:44.093500    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc" podStartSLOduration=1.093482668 podStartE2EDuration="1.093482668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:07:44.090194199 +0000 UTC m=+1762.401893656" watchObservedRunningTime="2026-07-14 20:07:44.093482668 +0000 UTC m=+1762.405182124"
Jul 14 20:07:45.083006 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:45.082963    2581 generic.go:356] "Generic (PLEG): container finished" podID="dbb4aa24-aeed-4145-bd9b-62da3c95993c" containerID="b3a0b84677950e97a659ae7326c15d3ac296d9a0b502b8262ed3e4423e8ca514" exitCode=0
Jul 14 20:07:45.083448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:45.083052    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc" event={"ID":"dbb4aa24-aeed-4145-bd9b-62da3c95993c","Type":"ContainerDied","Data":"b3a0b84677950e97a659ae7326c15d3ac296d9a0b502b8262ed3e4423e8ca514"}
Jul 14 20:07:46.246261 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:46.246231    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"
Jul 14 20:07:46.294602 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:46.294550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb4aa24-aeed-4145-bd9b-62da3c95993c-maas-api-service-ca\") pod \"dbb4aa24-aeed-4145-bd9b-62da3c95993c\" (UID: \"dbb4aa24-aeed-4145-bd9b-62da3c95993c\") "
Jul 14 20:07:46.294980 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:46.294941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb4aa24-aeed-4145-bd9b-62da3c95993c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dbb4aa24-aeed-4145-bd9b-62da3c95993c" (UID: "dbb4aa24-aeed-4145-bd9b-62da3c95993c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:07:46.396274 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:46.396159    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb4aa24-aeed-4145-bd9b-62da3c95993c-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:07:47.096577 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:47.096546    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc"
Jul 14 20:07:47.096577 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:47.096551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc" event={"ID":"dbb4aa24-aeed-4145-bd9b-62da3c95993c","Type":"ContainerDied","Data":"3c10fa88ada3dc0743512a2f8d397d4d9278729a998476a8046ed4a99308053c"}
Jul 14 20:07:47.096799 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:47.096611    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c10fa88ada3dc0743512a2f8d397d4d9278729a998476a8046ed4a99308053c"
Jul 14 20:07:57.034585 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:07:57.034545    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:08:08.349751 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:08.349710    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"]
Jul 14 20:08:08.350185 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:08.349958    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy" containerID="cri-o://773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c" gracePeriod=30
Jul 14 20:08:13.620906 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.620834    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:08:13.659509 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926646fb-eaca-4fcf-ad59-3f82e89eb851-istiod-ca-cert\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.659700 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659558    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-token\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.659700 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-data\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.659834 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659738    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgtmx\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-kube-api-access-fgtmx\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.659834 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-envoy\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.659942 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-certs\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.659942 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659867    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/926646fb-eaca-4fcf-ad59-3f82e89eb851-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:08:13.659942 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659898    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-socket\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.659942 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659932    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-podinfo\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.660170 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.659956    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-credential-socket\") pod \"926646fb-eaca-4fcf-ad59-3f82e89eb851\" (UID: \"926646fb-eaca-4fcf-ad59-3f82e89eb851\") "
Jul 14 20:08:13.660527 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.660308    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926646fb-eaca-4fcf-ad59-3f82e89eb851-istiod-ca-cert\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.660772 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.660736    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:08:13.660865 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.660789    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:08:13.660865 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.660799    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-data" (OuterVolumeSpecName: "istio-data") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:08:13.660984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.660963    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:08:13.662274 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.662246    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-token" (OuterVolumeSpecName: "istio-token") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:08:13.662457 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.662365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:08:13.662798 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.662757    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-kube-api-access-fgtmx" (OuterVolumeSpecName: "kube-api-access-fgtmx") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "kube-api-access-fgtmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:08:13.663015 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.662994    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "926646fb-eaca-4fcf-ad59-3f82e89eb851" (UID: "926646fb-eaca-4fcf-ad59-3f82e89eb851"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 20:08:13.761580 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761548    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-data\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.761580 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761575    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgtmx\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-kube-api-access-fgtmx\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.761580 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761585    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-envoy\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.761822 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761595    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.761822 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761603    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-workload-socket\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.761822 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761611    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-podinfo\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.761822 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761619    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926646fb-eaca-4fcf-ad59-3f82e89eb851-credential-socket\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:13.761822 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:13.761627    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926646fb-eaca-4fcf-ad59-3f82e89eb851-istio-token\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:14.270887 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.270842    2581 generic.go:356] "Generic (PLEG): container finished" podID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerID="773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c" exitCode=0
Jul 14 20:08:14.271081 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.270891    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"
Jul 14 20:08:14.271081 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.270922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" event={"ID":"926646fb-eaca-4fcf-ad59-3f82e89eb851","Type":"ContainerDied","Data":"773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c"}
Jul 14 20:08:14.271081 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.270987    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj" event={"ID":"926646fb-eaca-4fcf-ad59-3f82e89eb851","Type":"ContainerDied","Data":"b3c8efcc788c97c7a09c70d51775ff2693958149fe5425e6f448e01474b51849"}
Jul 14 20:08:14.271081 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.271015    2581 scope.go:117] "RemoveContainer" containerID="773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c"
Jul 14 20:08:14.282213 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.282193    2581 scope.go:117] "RemoveContainer" containerID="e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72"
Jul 14 20:08:14.292863 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.292836    2581 scope.go:117] "RemoveContainer" containerID="773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c"
Jul 14 20:08:14.293194 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:08:14.293174    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c\": container with ID starting with 773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c not found: ID does not exist" containerID="773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c"
Jul 14 20:08:14.293275 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.293210    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c"} err="failed to get container status \"773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c\": rpc error: code = NotFound desc = could not find container \"773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c\": container with ID starting with 773189c76c064eea2e327059efcacc6b2e11bc1b15fd9434730695cb1248000c not found: ID does not exist"
Jul 14 20:08:14.293275 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.293238    2581 scope.go:117] "RemoveContainer" containerID="e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72"
Jul 14 20:08:14.293615 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:08:14.293589    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72\": container with ID starting with e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72 not found: ID does not exist" containerID="e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72"
Jul 14 20:08:14.293671 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.293625    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72"} err="failed to get container status \"e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72\": rpc error: code = NotFound desc = could not find container \"e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72\": container with ID starting with e90073558e498e2b9f8767bdc718b065ae1ec14589caa0f6dbbb9ddcec132b72 not found: ID does not exist"
Jul 14 20:08:14.297913 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.297877    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"]
Jul 14 20:08:14.302353 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.302329    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ffd1ab16-openshift-default-787757b4bf-ln6fj"]
Jul 14 20:08:14.785395 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.785358    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"]
Jul 14 20:08:14.785833 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.785793    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb4aa24-aeed-4145-bd9b-62da3c95993c" containerName="revoke-keys"
Jul 14 20:08:14.785833 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.785806    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb4aa24-aeed-4145-bd9b-62da3c95993c" containerName="revoke-keys"
Jul 14 20:08:14.785833 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.785815    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy"
Jul 14 20:08:14.785833 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.785821    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy"
Jul 14 20:08:14.785985 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.785910    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy"
Jul 14 20:08:14.785985 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.785917    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb4aa24-aeed-4145-bd9b-62da3c95993c" containerName="revoke-keys"
Jul 14 20:08:14.788846 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.788823    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"
Jul 14 20:08:14.791373 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.791351    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:08:14.791509 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.791494    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:08:14.797983 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.797958    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"]
Jul 14 20:08:14.872683 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.872644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws\" (UID: \"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"
Jul 14 20:08:14.974256 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.974211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws\" (UID: \"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"
Jul 14 20:08:14.974947 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:14.974925    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws\" (UID: \"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"
Jul 14 20:08:15.101234 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:15.101135    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"
Jul 14 20:08:15.341090 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:15.341047    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" path="/var/lib/kubelet/pods/926646fb-eaca-4fcf-ad59-3f82e89eb851/volumes"
Jul 14 20:08:15.461501 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:15.461471    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"]
Jul 14 20:08:15.464030 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:08:15.463998    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b8de9b_f2b3_4e5e_a853_3a8122da2bd8.slice/crio-d28ffd6d8eeb5ebdc1e687b74f1c561021288edec47c2ef5685ad265b51307ee WatchSource:0}: Error finding container d28ffd6d8eeb5ebdc1e687b74f1c561021288edec47c2ef5685ad265b51307ee: Status 404 returned error can't find the container with id d28ffd6d8eeb5ebdc1e687b74f1c561021288edec47c2ef5685ad265b51307ee
Jul 14 20:08:16.290137 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:16.290092    2581 generic.go:356] "Generic (PLEG): container finished" podID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerID="ab959f0c2d53da2ea52b5c9fd0174a6db984b34e4a2ef336c2962c2bae0276e6" exitCode=7
Jul 14 20:08:16.290555 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:16.290184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws" event={"ID":"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8","Type":"ContainerDied","Data":"ab959f0c2d53da2ea52b5c9fd0174a6db984b34e4a2ef336c2962c2bae0276e6"}
Jul 14 20:08:16.290555 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:16.290251    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws" event={"ID":"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8","Type":"ContainerStarted","Data":"d28ffd6d8eeb5ebdc1e687b74f1c561021288edec47c2ef5685ad265b51307ee"}
Jul 14 20:08:16.290555 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:16.290386    2581 scope.go:117] "RemoveContainer" containerID="ab959f0c2d53da2ea52b5c9fd0174a6db984b34e4a2ef336c2962c2bae0276e6"
Jul 14 20:08:17.297697 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:17.297661    2581 generic.go:356] "Generic (PLEG): container finished" podID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerID="b7cded4cedd9c71f1bdbe8a19c76084c6eed57963f10956d1fbf1385ff0ddc48" exitCode=7
Jul 14 20:08:17.298197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:17.297753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws" event={"ID":"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8","Type":"ContainerDied","Data":"b7cded4cedd9c71f1bdbe8a19c76084c6eed57963f10956d1fbf1385ff0ddc48"}
Jul 14 20:08:17.298197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:17.297837    2581 scope.go:117] "RemoveContainer" containerID="ab959f0c2d53da2ea52b5c9fd0174a6db984b34e4a2ef336c2962c2bae0276e6"
Jul 14 20:08:17.298197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:17.297978    2581 scope.go:117] "RemoveContainer" containerID="b7cded4cedd9c71f1bdbe8a19c76084c6eed57963f10956d1fbf1385ff0ddc48"
Jul 14 20:08:17.298197 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:08:17.298185    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws_odh-ai-gateway-infra(c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8"
Jul 14 20:08:17.849345 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:17.849305    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:08:18.306263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:18.306238    2581 scope.go:117] "RemoveContainer" containerID="b7cded4cedd9c71f1bdbe8a19c76084c6eed57963f10956d1fbf1385ff0ddc48"
Jul 14 20:08:18.306688 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:08:18.306454    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws_odh-ai-gateway-infra(c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8"
Jul 14 20:08:23.379745 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:23.379709    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:08:23.388140 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:23.388115    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:08:31.332413 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:31.332377    2581 scope.go:117] "RemoveContainer" containerID="b7cded4cedd9c71f1bdbe8a19c76084c6eed57963f10956d1fbf1385ff0ddc48"
Jul 14 20:08:32.395373 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:32.395332    2581 generic.go:356] "Generic (PLEG): container finished" podID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerID="ae330abd78b5f50f0f2e3ff1b20f6aca1befccf197c1c102229f4ee22a0f5bfa" exitCode=0
Jul 14 20:08:32.395865 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:32.395439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws" event={"ID":"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8","Type":"ContainerDied","Data":"ae330abd78b5f50f0f2e3ff1b20f6aca1befccf197c1c102229f4ee22a0f5bfa"}
Jul 14 20:08:32.395865 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:32.395545    2581 scope.go:117] "RemoveContainer" containerID="b7cded4cedd9c71f1bdbe8a19c76084c6eed57963f10956d1fbf1385ff0ddc48"
Jul 14 20:08:33.425357 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:33.425323    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"]
Jul 14 20:08:33.546502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:33.546472    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"
Jul 14 20:08:33.647250 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:33.647214    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8-maas-api-service-ca\") pod \"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8\" (UID: \"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8\") "
Jul 14 20:08:33.647649 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:33.647621    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" (UID: "c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:08:33.748391 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:33.748352    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:08:34.411785 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:34.411750    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"
Jul 14 20:08:34.411785 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:34.411762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws" event={"ID":"c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8","Type":"ContainerDied","Data":"d28ffd6d8eeb5ebdc1e687b74f1c561021288edec47c2ef5685ad265b51307ee"}
Jul 14 20:08:34.412018 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:34.411811    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d28ffd6d8eeb5ebdc1e687b74f1c561021288edec47c2ef5685ad265b51307ee"
Jul 14 20:08:34.446060 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:34.446013    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"]
Jul 14 20:08:34.449069 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:34.449038    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-lw9ws"]
Jul 14 20:08:35.340133 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:08:35.340094    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" path="/var/lib/kubelet/pods/c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8/volumes"
Jul 14 20:09:04.834748 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.834706    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"]
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835146    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835156    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835166    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy"
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835171    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy"
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835189    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835196    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835204    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.835263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835210    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.835568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835280    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="926646fb-eaca-4fcf-ad59-3f82e89eb851" containerName="istio-proxy"
Jul 14 20:09:04.835568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835287    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.835568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.835295    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:04.838486 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.838466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"
Jul 14 20:09:04.841991 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.841965    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:09:04.842112 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.841965    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:09:04.847807 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.847778    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"]
Jul 14 20:09:04.935085 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:04.935034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca44855d-0bbb-4fa9-9cb2-2247f7ff8416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh\" (UID: \"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"
Jul 14 20:09:05.035965 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:05.035924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca44855d-0bbb-4fa9-9cb2-2247f7ff8416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh\" (UID: \"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"
Jul 14 20:09:05.036661 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:05.036642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca44855d-0bbb-4fa9-9cb2-2247f7ff8416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh\" (UID: \"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"
Jul 14 20:09:05.151326 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:05.151216    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"
Jul 14 20:09:05.528513 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:05.528485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"]
Jul 14 20:09:05.531015 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:09:05.530990    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca44855d_0bbb_4fa9_9cb2_2247f7ff8416.slice/crio-0dff74f8fa8dd51437310f3851696350110cec30fb79739a7d45a0fcaf6721ae WatchSource:0}: Error finding container 0dff74f8fa8dd51437310f3851696350110cec30fb79739a7d45a0fcaf6721ae: Status 404 returned error can't find the container with id 0dff74f8fa8dd51437310f3851696350110cec30fb79739a7d45a0fcaf6721ae
Jul 14 20:09:05.598211 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:05.598169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh" event={"ID":"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416","Type":"ContainerStarted","Data":"0dff74f8fa8dd51437310f3851696350110cec30fb79739a7d45a0fcaf6721ae"}
Jul 14 20:09:06.606579 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:06.606471    2581 generic.go:356] "Generic (PLEG): container finished" podID="ca44855d-0bbb-4fa9-9cb2-2247f7ff8416" containerID="4344d1126400ad577ab0dc6980ffa2973a1245837f6e6119af9ff79f9262b461" exitCode=0
Jul 14 20:09:06.606579 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:06.606525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh" event={"ID":"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416","Type":"ContainerDied","Data":"4344d1126400ad577ab0dc6980ffa2973a1245837f6e6119af9ff79f9262b461"}
Jul 14 20:09:07.766307 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:07.766273    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"
Jul 14 20:09:07.858643 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:07.858602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca44855d-0bbb-4fa9-9cb2-2247f7ff8416-maas-api-service-ca\") pod \"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416\" (UID: \"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416\") "
Jul 14 20:09:07.859018 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:07.858997    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca44855d-0bbb-4fa9-9cb2-2247f7ff8416-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca44855d-0bbb-4fa9-9cb2-2247f7ff8416" (UID: "ca44855d-0bbb-4fa9-9cb2-2247f7ff8416"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:09:07.959365 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:07.959333    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca44855d-0bbb-4fa9-9cb2-2247f7ff8416-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:09:08.619802 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:08.619773    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh"
Jul 14 20:09:08.619976 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:08.619817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh" event={"ID":"ca44855d-0bbb-4fa9-9cb2-2247f7ff8416","Type":"ContainerDied","Data":"0dff74f8fa8dd51437310f3851696350110cec30fb79739a7d45a0fcaf6721ae"}
Jul 14 20:09:08.619976 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:08.619870    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dff74f8fa8dd51437310f3851696350110cec30fb79739a7d45a0fcaf6721ae"
Jul 14 20:09:18.039177 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:18.039132    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:09:27.988536 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.988494    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"]
Jul 14 20:09:27.988998 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.988984    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca44855d-0bbb-4fa9-9cb2-2247f7ff8416" containerName="revoke-keys"
Jul 14 20:09:27.989044 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.989003    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca44855d-0bbb-4fa9-9cb2-2247f7ff8416" containerName="revoke-keys"
Jul 14 20:09:27.989105 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.989096    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca44855d-0bbb-4fa9-9cb2-2247f7ff8416" containerName="revoke-keys"
Jul 14 20:09:27.989142 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.989111    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2b8de9b-f2b3-4e5e-a853-3a8122da2bd8" containerName="revoke-keys"
Jul 14 20:09:27.992385 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.992363    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"
Jul 14 20:09:27.994760 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.994741    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:09:27.994838 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:27.994808    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:09:28.000077 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:28.000047    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"]
Jul 14 20:09:28.051555 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:28.051507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57d7c27-9c8a-4097-a15a-ca01fc9c2075-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh\" (UID: \"b57d7c27-9c8a-4097-a15a-ca01fc9c2075\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"
Jul 14 20:09:28.153077 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:28.153037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57d7c27-9c8a-4097-a15a-ca01fc9c2075-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh\" (UID: \"b57d7c27-9c8a-4097-a15a-ca01fc9c2075\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"
Jul 14 20:09:28.153809 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:28.153788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57d7c27-9c8a-4097-a15a-ca01fc9c2075-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh\" (UID: \"b57d7c27-9c8a-4097-a15a-ca01fc9c2075\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"
Jul 14 20:09:28.305681 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:28.305571    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"
Jul 14 20:09:28.450828 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:28.450800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"]
Jul 14 20:09:28.452712 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:09:28.452683    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57d7c27_9c8a_4097_a15a_ca01fc9c2075.slice/crio-a1c728a762820ab4cfd0b1e56c70e43d427b2ae9ca2735623ef5b40be73d5bb0 WatchSource:0}: Error finding container a1c728a762820ab4cfd0b1e56c70e43d427b2ae9ca2735623ef5b40be73d5bb0: Status 404 returned error can't find the container with id a1c728a762820ab4cfd0b1e56c70e43d427b2ae9ca2735623ef5b40be73d5bb0
Jul 14 20:09:28.743486 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:28.743440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" event={"ID":"b57d7c27-9c8a-4097-a15a-ca01fc9c2075","Type":"ContainerStarted","Data":"a1c728a762820ab4cfd0b1e56c70e43d427b2ae9ca2735623ef5b40be73d5bb0"}
Jul 14 20:09:29.751192 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:29.751159    2581 generic.go:356] "Generic (PLEG): container finished" podID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerID="53ca559588d89cbfe9c003e54243b8f03dc1ae2485de299fbb47fc0fb5c47fbc" exitCode=7
Jul 14 20:09:29.751741 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:29.751242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" event={"ID":"b57d7c27-9c8a-4097-a15a-ca01fc9c2075","Type":"ContainerDied","Data":"53ca559588d89cbfe9c003e54243b8f03dc1ae2485de299fbb47fc0fb5c47fbc"}
Jul 14 20:09:29.751741 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:29.751504    2581 scope.go:117] "RemoveContainer" containerID="53ca559588d89cbfe9c003e54243b8f03dc1ae2485de299fbb47fc0fb5c47fbc"
Jul 14 20:09:30.229200 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:30.229166    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:09:30.759363 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:30.759326    2581 generic.go:356] "Generic (PLEG): container finished" podID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerID="094a07a69d2dd7b87f4c2071a300a983279a1c0fff5e44a1899b279a7d55f06b" exitCode=7
Jul 14 20:09:30.759933 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:30.759438    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" event={"ID":"b57d7c27-9c8a-4097-a15a-ca01fc9c2075","Type":"ContainerDied","Data":"094a07a69d2dd7b87f4c2071a300a983279a1c0fff5e44a1899b279a7d55f06b"}
Jul 14 20:09:30.759933 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:30.759508    2581 scope.go:117] "RemoveContainer" containerID="53ca559588d89cbfe9c003e54243b8f03dc1ae2485de299fbb47fc0fb5c47fbc"
Jul 14 20:09:30.759933 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:30.759716    2581 scope.go:117] "RemoveContainer" containerID="094a07a69d2dd7b87f4c2071a300a983279a1c0fff5e44a1899b279a7d55f06b"
Jul 14 20:09:30.760074 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:09:30.759981    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh_odh-ai-gateway-infra(b57d7c27-9c8a-4097-a15a-ca01fc9c2075)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075"
Jul 14 20:09:31.767244 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:31.767212    2581 scope.go:117] "RemoveContainer" containerID="094a07a69d2dd7b87f4c2071a300a983279a1c0fff5e44a1899b279a7d55f06b"
Jul 14 20:09:31.767702 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:09:31.767485    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh_odh-ai-gateway-infra(b57d7c27-9c8a-4097-a15a-ca01fc9c2075)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075"
Jul 14 20:09:45.332599 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:45.332508    2581 scope.go:117] "RemoveContainer" containerID="094a07a69d2dd7b87f4c2071a300a983279a1c0fff5e44a1899b279a7d55f06b"
Jul 14 20:09:45.855879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:45.855837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" event={"ID":"b57d7c27-9c8a-4097-a15a-ca01fc9c2075","Type":"ContainerStarted","Data":"e60cf022a68bc3a3228c4c6a15fdc4355ac5cc10f0343cbae9908029bf9157e8"}
Jul 14 20:09:45.877188 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:45.877125    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" podStartSLOduration=18.877106128 podStartE2EDuration="18.877106128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:09:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:09:45.874000659 +0000 UTC m=+1884.185700119" watchObservedRunningTime="2026-07-14 20:09:45.877106128 +0000 UTC m=+1884.188805562"
Jul 14 20:09:46.864219 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:46.864183    2581 generic.go:356] "Generic (PLEG): container finished" podID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerID="e60cf022a68bc3a3228c4c6a15fdc4355ac5cc10f0343cbae9908029bf9157e8" exitCode=0
Jul 14 20:09:46.864665 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:46.864253    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" event={"ID":"b57d7c27-9c8a-4097-a15a-ca01fc9c2075","Type":"ContainerDied","Data":"e60cf022a68bc3a3228c4c6a15fdc4355ac5cc10f0343cbae9908029bf9157e8"}
Jul 14 20:09:46.864665 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:46.864315    2581 scope.go:117] "RemoveContainer" containerID="094a07a69d2dd7b87f4c2071a300a983279a1c0fff5e44a1899b279a7d55f06b"
Jul 14 20:09:46.893571 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:46.893538    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"]
Jul 14 20:09:48.020719 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.020687    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"
Jul 14 20:09:48.043862 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.043829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57d7c27-9c8a-4097-a15a-ca01fc9c2075-maas-api-service-ca\") pod \"b57d7c27-9c8a-4097-a15a-ca01fc9c2075\" (UID: \"b57d7c27-9c8a-4097-a15a-ca01fc9c2075\") "
Jul 14 20:09:48.044196 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.044172    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b57d7c27-9c8a-4097-a15a-ca01fc9c2075-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b57d7c27-9c8a-4097-a15a-ca01fc9c2075" (UID: "b57d7c27-9c8a-4097-a15a-ca01fc9c2075"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:09:48.144992 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.144882    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57d7c27-9c8a-4097-a15a-ca01fc9c2075-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:09:48.879644 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.879609    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"
Jul 14 20:09:48.879846 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.879642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh" event={"ID":"b57d7c27-9c8a-4097-a15a-ca01fc9c2075","Type":"ContainerDied","Data":"a1c728a762820ab4cfd0b1e56c70e43d427b2ae9ca2735623ef5b40be73d5bb0"}
Jul 14 20:09:48.879846 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.879701    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1c728a762820ab4cfd0b1e56c70e43d427b2ae9ca2735623ef5b40be73d5bb0"
Jul 14 20:09:48.912279 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.912244    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"]
Jul 14 20:09:48.920366 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:48.920331    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-bn2bh"]
Jul 14 20:09:49.338951 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:09:49.338916    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" path="/var/lib/kubelet/pods/b57d7c27-9c8a-4097-a15a-ca01fc9c2075/volumes"
Jul 14 20:10:28.038365 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.038322    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"]
Jul 14 20:10:28.039026 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039001    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.039117 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039029    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.039117 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039044    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.039117 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039052    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.039117 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039063    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.039117 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039071    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.039281 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039190    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.039281 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.039204    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:28.042366 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.042339    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"
Jul 14 20:10:28.045090 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.045069    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:10:28.045206 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.045186    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:10:28.051108 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.051080    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"]
Jul 14 20:10:28.122262 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.122219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp\" (UID: \"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"
Jul 14 20:10:28.223919 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.223873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp\" (UID: \"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"
Jul 14 20:10:28.224595 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.224575    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp\" (UID: \"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"
Jul 14 20:10:28.355218 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.355122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"
Jul 14 20:10:28.502910 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:28.502873    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"]
Jul 14 20:10:28.505020 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:10:28.504985    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf4c3d5_da86_4ca5_a7c2_f9bb4a1788c3.slice/crio-0ab49596e16de461fcecffc7c3cd28714342135a0a74f8177eb9573d14a39e34 WatchSource:0}: Error finding container 0ab49596e16de461fcecffc7c3cd28714342135a0a74f8177eb9573d14a39e34: Status 404 returned error can't find the container with id 0ab49596e16de461fcecffc7c3cd28714342135a0a74f8177eb9573d14a39e34
Jul 14 20:10:29.126609 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:29.126571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp" event={"ID":"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3","Type":"ContainerStarted","Data":"ae4969d072cfd47f4a7582dc90ebdbaa7e07bd9464ee959929d9740b966b4e97"}
Jul 14 20:10:29.127010 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:29.126662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp" event={"ID":"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3","Type":"ContainerStarted","Data":"0ab49596e16de461fcecffc7c3cd28714342135a0a74f8177eb9573d14a39e34"}
Jul 14 20:10:29.143296 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:29.143227    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp" podStartSLOduration=1.143210965 podStartE2EDuration="1.143210965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:10:29.141686824 +0000 UTC m=+1927.453386279" watchObservedRunningTime="2026-07-14 20:10:29.143210965 +0000 UTC m=+1927.454910429"
Jul 14 20:10:30.133877 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:30.133837    2581 generic.go:356] "Generic (PLEG): container finished" podID="acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3" containerID="ae4969d072cfd47f4a7582dc90ebdbaa7e07bd9464ee959929d9740b966b4e97" exitCode=0
Jul 14 20:10:30.134292 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:30.133902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp" event={"ID":"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3","Type":"ContainerDied","Data":"ae4969d072cfd47f4a7582dc90ebdbaa7e07bd9464ee959929d9740b966b4e97"}
Jul 14 20:10:31.294248 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:31.294222    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"
Jul 14 20:10:31.350976 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:31.350940    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3-maas-api-service-ca\") pod \"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3\" (UID: \"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3\") "
Jul 14 20:10:31.351367 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:31.351331    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3" (UID: "acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:10:31.451781 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:31.451747    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:10:32.147093 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:32.147060    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp"
Jul 14 20:10:32.147291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:32.147095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp" event={"ID":"acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3","Type":"ContainerDied","Data":"0ab49596e16de461fcecffc7c3cd28714342135a0a74f8177eb9573d14a39e34"}
Jul 14 20:10:32.147291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:32.147152    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ab49596e16de461fcecffc7c3cd28714342135a0a74f8177eb9573d14a39e34"
Jul 14 20:10:41.449072 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:41.449026    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:10:55.953988 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.953953    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"]
Jul 14 20:10:55.954448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.954383    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3" containerName="revoke-keys"
Jul 14 20:10:55.954448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.954395    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3" containerName="revoke-keys"
Jul 14 20:10:55.954532 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.954490    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57d7c27-9c8a-4097-a15a-ca01fc9c2075" containerName="revoke-keys"
Jul 14 20:10:55.954532 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.954498    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3" containerName="revoke-keys"
Jul 14 20:10:55.957725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.957706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:55.962609 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.962582    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tfc6h\""
Jul 14 20:10:55.963689 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.963670    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 20:10:55.963766 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.963747    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:10:55.967633 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.967603    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-8983c49d-serving-cert\""
Jul 14 20:10:55.990451 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:55.990395    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"]
Jul 14 20:10:56.091748 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.091705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk25q\" (UniqueName: \"kubernetes.io/projected/fc16131f-1fe2-4773-8af9-0e24189f5eb3-kube-api-access-hk25q\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.091947 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.091758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc16131f-1fe2-4773-8af9-0e24189f5eb3-maas-api-tls\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.091947 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.091863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc16131f-1fe2-4773-8af9-0e24189f5eb3-openshift-service-ca\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.192810 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.192767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk25q\" (UniqueName: \"kubernetes.io/projected/fc16131f-1fe2-4773-8af9-0e24189f5eb3-kube-api-access-hk25q\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.192810 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.192818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc16131f-1fe2-4773-8af9-0e24189f5eb3-maas-api-tls\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.193111 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.192874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc16131f-1fe2-4773-8af9-0e24189f5eb3-openshift-service-ca\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.193642 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.193616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc16131f-1fe2-4773-8af9-0e24189f5eb3-openshift-service-ca\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.195703 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.195675    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc16131f-1fe2-4773-8af9-0e24189f5eb3-maas-api-tls\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.202847 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.202813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk25q\" (UniqueName: \"kubernetes.io/projected/fc16131f-1fe2-4773-8af9-0e24189f5eb3-kube-api-access-hk25q\") pod \"maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.270865 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.270761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:56.432929 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.432900    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"]
Jul 14 20:10:56.434754 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:10:56.434722    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc16131f_1fe2_4773_8af9_0e24189f5eb3.slice/crio-83acfde0d34cf26c998281eddf52dc0bfecc62f4b652123f6df916e61efd88ac WatchSource:0}: Error finding container 83acfde0d34cf26c998281eddf52dc0bfecc62f4b652123f6df916e61efd88ac: Status 404 returned error can't find the container with id 83acfde0d34cf26c998281eddf52dc0bfecc62f4b652123f6df916e61efd88ac
Jul 14 20:10:56.436013 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:56.435994    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 20:10:57.312062 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:57.312020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc" event={"ID":"fc16131f-1fe2-4773-8af9-0e24189f5eb3","Type":"ContainerStarted","Data":"83acfde0d34cf26c998281eddf52dc0bfecc62f4b652123f6df916e61efd88ac"}
Jul 14 20:10:58.320545 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:58.320437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc" event={"ID":"fc16131f-1fe2-4773-8af9-0e24189f5eb3","Type":"ContainerStarted","Data":"eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8"}
Jul 14 20:10:58.320545 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:58.320539    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:10:58.343728 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:10:58.343675    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc" podStartSLOduration=1.737592303 podStartE2EDuration="3.343658258s" totalImagesPullingTime="1.606065955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:10:58.340963838 +0000 UTC m=+1956.652663293" watchObservedRunningTime="2026-07-14 20:10:58.343658258 +0000 UTC m=+1956.655357714"
Jul 14 20:11:00.023129 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.023085    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"]
Jul 14 20:11:00.027041 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.027020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"
Jul 14 20:11:00.029821 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.029797    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:11:00.036474 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.036439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"]
Jul 14 20:11:00.131002 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.130967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0775451-0f7d-44b6-9ca9-d9549ddf7439-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk\" (UID: \"b0775451-0f7d-44b6-9ca9-d9549ddf7439\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"
Jul 14 20:11:00.231765 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.231727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0775451-0f7d-44b6-9ca9-d9549ddf7439-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk\" (UID: \"b0775451-0f7d-44b6-9ca9-d9549ddf7439\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"
Jul 14 20:11:00.232477 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.232452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0775451-0f7d-44b6-9ca9-d9549ddf7439-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk\" (UID: \"b0775451-0f7d-44b6-9ca9-d9549ddf7439\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"
Jul 14 20:11:00.339807 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.339713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"
Jul 14 20:11:00.692641 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:00.692606    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"]
Jul 14 20:11:00.694250 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:11:00.694223    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0775451_0f7d_44b6_9ca9_d9549ddf7439.slice/crio-6128107dc499118c60b38ac80b997e00e30a9ebec6ed2e8362ab6d95f5bcbc25 WatchSource:0}: Error finding container 6128107dc499118c60b38ac80b997e00e30a9ebec6ed2e8362ab6d95f5bcbc25: Status 404 returned error can't find the container with id 6128107dc499118c60b38ac80b997e00e30a9ebec6ed2e8362ab6d95f5bcbc25
Jul 14 20:11:01.348330 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:01.348235    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" event={"ID":"b0775451-0f7d-44b6-9ca9-d9549ddf7439","Type":"ContainerStarted","Data":"94aad3226b14b389be9589b6807fe70253b85754796cd253e638b5daf0ac13e7"}
Jul 14 20:11:01.348330 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:01.348298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" event={"ID":"b0775451-0f7d-44b6-9ca9-d9549ddf7439","Type":"ContainerStarted","Data":"6128107dc499118c60b38ac80b997e00e30a9ebec6ed2e8362ab6d95f5bcbc25"}
Jul 14 20:11:01.368432 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:01.368340    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" podStartSLOduration=1.368318669 podStartE2EDuration="1.368318669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:11:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:11:01.366981336 +0000 UTC m=+1959.678680791" watchObservedRunningTime="2026-07-14 20:11:01.368318669 +0000 UTC m=+1959.680018127"
Jul 14 20:11:02.357231 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:02.357189    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerID="94aad3226b14b389be9589b6807fe70253b85754796cd253e638b5daf0ac13e7" exitCode=7
Jul 14 20:11:02.357743 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:02.357250    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" event={"ID":"b0775451-0f7d-44b6-9ca9-d9549ddf7439","Type":"ContainerDied","Data":"94aad3226b14b389be9589b6807fe70253b85754796cd253e638b5daf0ac13e7"}
Jul 14 20:11:02.357743 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:02.357515    2581 scope.go:117] "RemoveContainer" containerID="94aad3226b14b389be9589b6807fe70253b85754796cd253e638b5daf0ac13e7"
Jul 14 20:11:03.365219 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:03.365116    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerID="5790e333d5b691fe1a0b1a934cbacce614a10a8c32f68b3f46670fa2c4204595" exitCode=7
Jul 14 20:11:03.365219 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:03.365195    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" event={"ID":"b0775451-0f7d-44b6-9ca9-d9549ddf7439","Type":"ContainerDied","Data":"5790e333d5b691fe1a0b1a934cbacce614a10a8c32f68b3f46670fa2c4204595"}
Jul 14 20:11:03.365779 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:03.365260    2581 scope.go:117] "RemoveContainer" containerID="94aad3226b14b389be9589b6807fe70253b85754796cd253e638b5daf0ac13e7"
Jul 14 20:11:03.365779 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:03.365497    2581 scope.go:117] "RemoveContainer" containerID="5790e333d5b691fe1a0b1a934cbacce614a10a8c32f68b3f46670fa2c4204595"
Jul 14 20:11:03.365779 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:11:03.365720    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk_odh-ai-gateway-infra(b0775451-0f7d-44b6-9ca9-d9549ddf7439)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439"
Jul 14 20:11:03.654069 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:03.653970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:11:04.332465 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:04.332436    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:11:04.376528 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:04.376502    2581 scope.go:117] "RemoveContainer" containerID="5790e333d5b691fe1a0b1a934cbacce614a10a8c32f68b3f46670fa2c4204595"
Jul 14 20:11:04.377020 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:11:04.376683    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk_odh-ai-gateway-infra(b0775451-0f7d-44b6-9ca9-d9549ddf7439)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439"
Jul 14 20:11:16.332085 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:16.332004    2581 scope.go:117] "RemoveContainer" containerID="5790e333d5b691fe1a0b1a934cbacce614a10a8c32f68b3f46670fa2c4204595"
Jul 14 20:11:17.463907 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:17.463871    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerID="0f3450558d593373d2f484481e8624809af7773029863ad88b7c593cb51c109f" exitCode=0
Jul 14 20:11:17.463907 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:17.463898    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" event={"ID":"b0775451-0f7d-44b6-9ca9-d9549ddf7439","Type":"ContainerDied","Data":"0f3450558d593373d2f484481e8624809af7773029863ad88b7c593cb51c109f"}
Jul 14 20:11:17.464430 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:17.463981    2581 scope.go:117] "RemoveContainer" containerID="5790e333d5b691fe1a0b1a934cbacce614a10a8c32f68b3f46670fa2c4204595"
Jul 14 20:11:18.500797 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:18.500762    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"]
Jul 14 20:11:18.628790 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:18.628760    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"
Jul 14 20:11:18.710368 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:18.710328    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0775451-0f7d-44b6-9ca9-d9549ddf7439-maas-api-service-ca\") pod \"b0775451-0f7d-44b6-9ca9-d9549ddf7439\" (UID: \"b0775451-0f7d-44b6-9ca9-d9549ddf7439\") "
Jul 14 20:11:18.710727 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:18.710697    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0775451-0f7d-44b6-9ca9-d9549ddf7439-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0775451-0f7d-44b6-9ca9-d9549ddf7439" (UID: "b0775451-0f7d-44b6-9ca9-d9549ddf7439"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:11:18.811964 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:18.811874    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0775451-0f7d-44b6-9ca9-d9549ddf7439-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:11:19.479997 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:19.479964    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"
Jul 14 20:11:19.480197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:19.480003    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk" event={"ID":"b0775451-0f7d-44b6-9ca9-d9549ddf7439","Type":"ContainerDied","Data":"6128107dc499118c60b38ac80b997e00e30a9ebec6ed2e8362ab6d95f5bcbc25"}
Jul 14 20:11:19.480197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:19.480051    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6128107dc499118c60b38ac80b997e00e30a9ebec6ed2e8362ab6d95f5bcbc25"
Jul 14 20:11:19.523892 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:19.523864    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"]
Jul 14 20:11:19.526459 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:19.526435    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-5r5pk"]
Jul 14 20:11:21.338632 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:21.338591    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" path="/var/lib/kubelet/pods/b0775451-0f7d-44b6-9ca9-d9549ddf7439/volumes"
Jul 14 20:11:50.079552 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.079517    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"]
Jul 14 20:11:50.080011 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.079965    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:11:50.080011 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.079976    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:11:50.080011 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.079995    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:11:50.080011 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.080001    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:11:50.080146 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.080066    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:11:50.080146 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.080074    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:11:50.083980 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.083961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"
Jul 14 20:11:50.086325 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.086297    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:11:50.091863 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.091835    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"]
Jul 14 20:11:50.208900 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.208854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89b4368-e7d5-49d1-8817-1112924b6efe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8983c49d-cslh8\" (UID: \"d89b4368-e7d5-49d1-8817-1112924b6efe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"
Jul 14 20:11:50.309842 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.309786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89b4368-e7d5-49d1-8817-1112924b6efe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8983c49d-cslh8\" (UID: \"d89b4368-e7d5-49d1-8817-1112924b6efe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"
Jul 14 20:11:50.310451 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.310432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89b4368-e7d5-49d1-8817-1112924b6efe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8983c49d-cslh8\" (UID: \"d89b4368-e7d5-49d1-8817-1112924b6efe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"
Jul 14 20:11:50.396971 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.396873    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"
Jul 14 20:11:50.546436 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:11:50.546214    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89b4368_e7d5_49d1_8817_1112924b6efe.slice/crio-d88fee9a32421cbba33d90d6f3c72b4738c8633fec99eaa7d6b5ed9b017ff9f2 WatchSource:0}: Error finding container d88fee9a32421cbba33d90d6f3c72b4738c8633fec99eaa7d6b5ed9b017ff9f2: Status 404 returned error can't find the container with id d88fee9a32421cbba33d90d6f3c72b4738c8633fec99eaa7d6b5ed9b017ff9f2
Jul 14 20:11:50.546615 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.546583    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"]
Jul 14 20:11:50.679765 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:50.679675    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8" event={"ID":"d89b4368-e7d5-49d1-8817-1112924b6efe","Type":"ContainerStarted","Data":"d88fee9a32421cbba33d90d6f3c72b4738c8633fec99eaa7d6b5ed9b017ff9f2"}
Jul 14 20:11:51.687706 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:51.687605    2581 generic.go:356] "Generic (PLEG): container finished" podID="d89b4368-e7d5-49d1-8817-1112924b6efe" containerID="4fb6f2d94f0292148bf976be7baaa380e41cf77629176cca9dbdb2149708a6e1" exitCode=0
Jul 14 20:11:51.687706 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:51.687649    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8" event={"ID":"d89b4368-e7d5-49d1-8817-1112924b6efe","Type":"ContainerDied","Data":"4fb6f2d94f0292148bf976be7baaa380e41cf77629176cca9dbdb2149708a6e1"}
Jul 14 20:11:52.842177 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:52.842149    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"
Jul 14 20:11:52.930572 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:52.930535    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89b4368-e7d5-49d1-8817-1112924b6efe-maas-api-service-ca\") pod \"d89b4368-e7d5-49d1-8817-1112924b6efe\" (UID: \"d89b4368-e7d5-49d1-8817-1112924b6efe\") "
Jul 14 20:11:52.930916 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:52.930892    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d89b4368-e7d5-49d1-8817-1112924b6efe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d89b4368-e7d5-49d1-8817-1112924b6efe" (UID: "d89b4368-e7d5-49d1-8817-1112924b6efe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:11:53.031766 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:53.031732    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89b4368-e7d5-49d1-8817-1112924b6efe-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:11:53.701612 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:53.701582    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8"
Jul 14 20:11:53.701791 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:53.701608    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8983c49d-cslh8" event={"ID":"d89b4368-e7d5-49d1-8817-1112924b6efe","Type":"ContainerDied","Data":"d88fee9a32421cbba33d90d6f3c72b4738c8633fec99eaa7d6b5ed9b017ff9f2"}
Jul 14 20:11:53.701791 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:11:53.701660    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d88fee9a32421cbba33d90d6f3c72b4738c8633fec99eaa7d6b5ed9b017ff9f2"
Jul 14 20:12:00.672073 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:00.672030    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"]
Jul 14 20:12:00.674801 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:00.672272    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc" podUID="fc16131f-1fe2-4773-8af9-0e24189f5eb3" containerName="maas-api" containerID="cri-o://eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8" gracePeriod=30
Jul 14 20:12:00.947272 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:00.947238    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:12:01.005986 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.005943    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc16131f-1fe2-4773-8af9-0e24189f5eb3-maas-api-tls\") pod \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") "
Jul 14 20:12:01.005986 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.005995    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc16131f-1fe2-4773-8af9-0e24189f5eb3-openshift-service-ca\") pod \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") "
Jul 14 20:12:01.006253 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.006044    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk25q\" (UniqueName: \"kubernetes.io/projected/fc16131f-1fe2-4773-8af9-0e24189f5eb3-kube-api-access-hk25q\") pod \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\" (UID: \"fc16131f-1fe2-4773-8af9-0e24189f5eb3\") "
Jul 14 20:12:01.006526 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.006491    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc16131f-1fe2-4773-8af9-0e24189f5eb3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc16131f-1fe2-4773-8af9-0e24189f5eb3" (UID: "fc16131f-1fe2-4773-8af9-0e24189f5eb3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:12:01.008384 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.008352    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc16131f-1fe2-4773-8af9-0e24189f5eb3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc16131f-1fe2-4773-8af9-0e24189f5eb3" (UID: "fc16131f-1fe2-4773-8af9-0e24189f5eb3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:12:01.008520 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.008351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc16131f-1fe2-4773-8af9-0e24189f5eb3-kube-api-access-hk25q" (OuterVolumeSpecName: "kube-api-access-hk25q") pod "fc16131f-1fe2-4773-8af9-0e24189f5eb3" (UID: "fc16131f-1fe2-4773-8af9-0e24189f5eb3"). InnerVolumeSpecName "kube-api-access-hk25q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:12:01.107149 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.107104    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc16131f-1fe2-4773-8af9-0e24189f5eb3-maas-api-tls\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:12:01.107149 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.107145    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc16131f-1fe2-4773-8af9-0e24189f5eb3-openshift-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:12:01.107149 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.107155    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk25q\" (UniqueName: \"kubernetes.io/projected/fc16131f-1fe2-4773-8af9-0e24189f5eb3-kube-api-access-hk25q\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:12:01.752870 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.752830    2581 generic.go:356] "Generic (PLEG): container finished" podID="fc16131f-1fe2-4773-8af9-0e24189f5eb3" containerID="eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8" exitCode=0
Jul 14 20:12:01.752870 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.752870    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"
Jul 14 20:12:01.753445 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.752914    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc" event={"ID":"fc16131f-1fe2-4773-8af9-0e24189f5eb3","Type":"ContainerDied","Data":"eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8"}
Jul 14 20:12:01.753445 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.752972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc" event={"ID":"fc16131f-1fe2-4773-8af9-0e24189f5eb3","Type":"ContainerDied","Data":"83acfde0d34cf26c998281eddf52dc0bfecc62f4b652123f6df916e61efd88ac"}
Jul 14 20:12:01.753445 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.753002    2581 scope.go:117] "RemoveContainer" containerID="eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8"
Jul 14 20:12:01.765447 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.765377    2581 scope.go:117] "RemoveContainer" containerID="eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8"
Jul 14 20:12:01.765924 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:12:01.765899    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8\": container with ID starting with eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8 not found: ID does not exist" containerID="eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8"
Jul 14 20:12:01.766022 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.765937    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8"} err="failed to get container status \"eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8\": rpc error: code = NotFound desc = could not find container \"eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8\": container with ID starting with eb591b0435f94a558562c4abc62db1e97cde65a5293ce8e53bc454ced53cfdf8 not found: ID does not exist"
Jul 14 20:12:01.770609 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.770574    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"]
Jul 14 20:12:01.774625 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:01.774589    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8983c49d-795cdfbf64-jf9kc"]
Jul 14 20:12:03.341071 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:03.341033    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc16131f-1fe2-4773-8af9-0e24189f5eb3" path="/var/lib/kubelet/pods/fc16131f-1fe2-4773-8af9-0e24189f5eb3/volumes"
Jul 14 20:12:03.341483 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:12:03.341329    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:13:23.422143 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:13:23.422114    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:13:23.431716 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:13:23.431689    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:13:23.633672 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:13:23.633638    2581 scope.go:117] "RemoveContainer" containerID="f6a24f8f78c7658c40c8692fe73c984d7c41c98b1c9c93b00ab38ff2f5d89e33"
Jul 14 20:13:40.238995 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:13:40.238954    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:13:49.536422 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:13:49.536359    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:14:01.741065 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:01.741022    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:14:08.441358 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441317    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"]
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441788    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441800    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441816    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc16131f-1fe2-4773-8af9-0e24189f5eb3" containerName="maas-api"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441821    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc16131f-1fe2-4773-8af9-0e24189f5eb3" containerName="maas-api"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441830    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89b4368-e7d5-49d1-8817-1112924b6efe" containerName="revoke-keys"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441836    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89b4368-e7d5-49d1-8817-1112924b6efe" containerName="revoke-keys"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441906    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0775451-0f7d-44b6-9ca9-d9549ddf7439" containerName="revoke-keys"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441917    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89b4368-e7d5-49d1-8817-1112924b6efe" containerName="revoke-keys"
Jul 14 20:14:08.442167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.441925    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc16131f-1fe2-4773-8af9-0e24189f5eb3" containerName="maas-api"
Jul 14 20:14:08.445247 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.445229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"
Jul 14 20:14:08.447950 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.447924    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:14:08.448076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.447955    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:14:08.453184 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.453156    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"]
Jul 14 20:14:08.537522 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.537468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-43646543-h42dq\" (UID: \"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"
Jul 14 20:14:08.638315 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.638273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-43646543-h42dq\" (UID: \"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"
Jul 14 20:14:08.638984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.638962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-43646543-h42dq\" (UID: \"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"
Jul 14 20:14:08.757900 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:08.757852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"
Jul 14 20:14:09.119873 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:09.119839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"]
Jul 14 20:14:09.121675 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:14:09.121650    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5a9a1a_1572_4ca8_a82e_0e40da19a9f5.slice/crio-c1b3313a31495b15112cffd633564a27500dfe3cffee2a034be3f18f71813718 WatchSource:0}: Error finding container c1b3313a31495b15112cffd633564a27500dfe3cffee2a034be3f18f71813718: Status 404 returned error can't find the container with id c1b3313a31495b15112cffd633564a27500dfe3cffee2a034be3f18f71813718
Jul 14 20:14:09.560096 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:09.560059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq" event={"ID":"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5","Type":"ContainerStarted","Data":"1845ceb274ec050f94e60f1c1066137636b8c2b96c5b861670d6ded8f1354484"}
Jul 14 20:14:09.560568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:09.560125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq" event={"ID":"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5","Type":"ContainerStarted","Data":"c1b3313a31495b15112cffd633564a27500dfe3cffee2a034be3f18f71813718"}
Jul 14 20:14:09.576214 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:09.576151    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq" podStartSLOduration=1.5761310750000002 podStartE2EDuration="1.576131075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:14:09.574251389 +0000 UTC m=+2147.885950845" watchObservedRunningTime="2026-07-14 20:14:09.576131075 +0000 UTC m=+2147.887830591"
Jul 14 20:14:10.569287 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:10.569248    2581 generic.go:356] "Generic (PLEG): container finished" podID="4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5" containerID="1845ceb274ec050f94e60f1c1066137636b8c2b96c5b861670d6ded8f1354484" exitCode=0
Jul 14 20:14:10.569715 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:10.569326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq" event={"ID":"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5","Type":"ContainerDied","Data":"1845ceb274ec050f94e60f1c1066137636b8c2b96c5b861670d6ded8f1354484"}
Jul 14 20:14:11.719725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:11.719696    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"
Jul 14 20:14:11.766742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:11.766702    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5-maas-api-service-ca\") pod \"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5\" (UID: \"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5\") "
Jul 14 20:14:11.767121 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:11.767093    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5" (UID: "4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:14:11.868278 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:11.868179    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:14:12.583371 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:12.583329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq" event={"ID":"4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5","Type":"ContainerDied","Data":"c1b3313a31495b15112cffd633564a27500dfe3cffee2a034be3f18f71813718"}
Jul 14 20:14:12.583371 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:12.583361    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43646543-h42dq"
Jul 14 20:14:12.583612 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:12.583396    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1b3313a31495b15112cffd633564a27500dfe3cffee2a034be3f18f71813718"
Jul 14 20:14:22.330471 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:14:22.330376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:15:00.144348 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.144304    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"]
Jul 14 20:15:00.144867 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.144772    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5" containerName="revoke-keys"
Jul 14 20:15:00.144867 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.144787    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5" containerName="revoke-keys"
Jul 14 20:15:00.144867 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.144865    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5" containerName="revoke-keys"
Jul 14 20:15:00.148040 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.148020    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4"]
Jul 14 20:15:00.148171 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.148148    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"
Jul 14 20:15:00.150792 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.150770    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:15:00.151760 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.151730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4"
Jul 14 20:15:00.155039 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.155016    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4"]
Jul 14 20:15:00.162652 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.162621    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"
Jul 14 20:15:00.162811 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.162786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"]
Jul 14 20:15:00.171869 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.171749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4"
Jul 14 20:15:00.328714 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.328678    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"]
Jul 14 20:15:00.330269 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:15:00.330233    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37815aee_acec_464a_bb97_33f9318dbc8a.slice/crio-cbd507138761d56acd4528e282015ba359e2bd06469595a5aa21bc5de3e8c44d WatchSource:0}: Error finding container cbd507138761d56acd4528e282015ba359e2bd06469595a5aa21bc5de3e8c44d: Status 404 returned error can't find the container with id cbd507138761d56acd4528e282015ba359e2bd06469595a5aa21bc5de3e8c44d
Jul 14 20:15:00.362501 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.362475    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4"]
Jul 14 20:15:00.376822 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:15:00.376797    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7555b08_4c25_4de4_b270_6dcab0971c05.slice/crio-0d8c18f4bb3227306b491660d889213aed7b37b1b9e793da9dc7c7a87b47011d WatchSource:0}: Error finding container 0d8c18f4bb3227306b491660d889213aed7b37b1b9e793da9dc7c7a87b47011d: Status 404 returned error can't find the container with id 0d8c18f4bb3227306b491660d889213aed7b37b1b9e793da9dc7c7a87b47011d
Jul 14 20:15:00.899077 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.899036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4" event={"ID":"d7555b08-4c25-4de4-b270-6dcab0971c05","Type":"ContainerStarted","Data":"bccb25db08a749b16d457604bc3f1456ab84cbebfbf5a7f74058e5a73aed89a3"}
Jul 14 20:15:00.899299 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.899104    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4" event={"ID":"d7555b08-4c25-4de4-b270-6dcab0971c05","Type":"ContainerStarted","Data":"0d8c18f4bb3227306b491660d889213aed7b37b1b9e793da9dc7c7a87b47011d"}
Jul 14 20:15:00.900639 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.900612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" event={"ID":"37815aee-acec-464a-bb97-33f9318dbc8a","Type":"ContainerStarted","Data":"11a107eb8de373700ca58e5b84db5d0f0284f120e0d876a91a270dff9097751e"}
Jul 14 20:15:00.900753 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.900658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" event={"ID":"37815aee-acec-464a-bb97-33f9318dbc8a","Type":"ContainerStarted","Data":"cbd507138761d56acd4528e282015ba359e2bd06469595a5aa21bc5de3e8c44d"}
Jul 14 20:15:00.923890 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.923834    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4" podStartSLOduration=0.923816754 podStartE2EDuration="923.816754ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:15:00.919859415 +0000 UTC m=+2199.231558871" watchObservedRunningTime="2026-07-14 20:15:00.923816754 +0000 UTC m=+2199.235516220"
Jul 14 20:15:00.949148 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:00.949100    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" podStartSLOduration=0.949080766 podStartE2EDuration="949.080766ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:15:00.945056233 +0000 UTC m=+2199.256755706" watchObservedRunningTime="2026-07-14 20:15:00.949080766 +0000 UTC m=+2199.260780222"
Jul 14 20:15:01.909544 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:01.909505    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7555b08-4c25-4de4-b270-6dcab0971c05" containerID="bccb25db08a749b16d457604bc3f1456ab84cbebfbf5a7f74058e5a73aed89a3" exitCode=0
Jul 14 20:15:01.910168 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:01.909797    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4" event={"ID":"d7555b08-4c25-4de4-b270-6dcab0971c05","Type":"ContainerDied","Data":"bccb25db08a749b16d457604bc3f1456ab84cbebfbf5a7f74058e5a73aed89a3"}
Jul 14 20:15:01.912581 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:01.912520    2581 generic.go:356] "Generic (PLEG): container finished" podID="37815aee-acec-464a-bb97-33f9318dbc8a" containerID="11a107eb8de373700ca58e5b84db5d0f0284f120e0d876a91a270dff9097751e" exitCode=7
Jul 14 20:15:01.912581 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:01.912580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" event={"ID":"37815aee-acec-464a-bb97-33f9318dbc8a","Type":"ContainerDied","Data":"11a107eb8de373700ca58e5b84db5d0f0284f120e0d876a91a270dff9097751e"}
Jul 14 20:15:01.912941 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:01.912922    2581 scope.go:117] "RemoveContainer" containerID="11a107eb8de373700ca58e5b84db5d0f0284f120e0d876a91a270dff9097751e"
Jul 14 20:15:02.922111 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:02.922002    2581 generic.go:356] "Generic (PLEG): container finished" podID="37815aee-acec-464a-bb97-33f9318dbc8a" containerID="6f439a20973e1461cece0d48fad5d489a4d46a18a70b0c37122e0546e3e16577" exitCode=7
Jul 14 20:15:02.922111 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:02.922097    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" event={"ID":"37815aee-acec-464a-bb97-33f9318dbc8a","Type":"ContainerDied","Data":"6f439a20973e1461cece0d48fad5d489a4d46a18a70b0c37122e0546e3e16577"}
Jul 14 20:15:02.922655 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:02.922170    2581 scope.go:117] "RemoveContainer" containerID="11a107eb8de373700ca58e5b84db5d0f0284f120e0d876a91a270dff9097751e"
Jul 14 20:15:02.922655 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:02.922337    2581 scope.go:117] "RemoveContainer" containerID="6f439a20973e1461cece0d48fad5d489a4d46a18a70b0c37122e0546e3e16577"
Jul 14 20:15:02.923042 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:15:02.922662    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8_odh-ai-gateway-infra(37815aee-acec-464a-bb97-33f9318dbc8a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" podUID="37815aee-acec-464a-bb97-33f9318dbc8a"
Jul 14 20:15:03.076353 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:03.076322    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4"
Jul 14 20:15:03.340852 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:03.340817    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:15:03.931040 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:03.931010    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4"
Jul 14 20:15:03.931040 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:03.931018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734335-vz9n4" event={"ID":"d7555b08-4c25-4de4-b270-6dcab0971c05","Type":"ContainerDied","Data":"0d8c18f4bb3227306b491660d889213aed7b37b1b9e793da9dc7c7a87b47011d"}
Jul 14 20:15:03.931556 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:03.931072    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d8c18f4bb3227306b491660d889213aed7b37b1b9e793da9dc7c7a87b47011d"
Jul 14 20:15:03.932957 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:03.932932    2581 scope.go:117] "RemoveContainer" containerID="6f439a20973e1461cece0d48fad5d489a4d46a18a70b0c37122e0546e3e16577"
Jul 14 20:15:03.933294 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:15:03.933264    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8_odh-ai-gateway-infra(37815aee-acec-464a-bb97-33f9318dbc8a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" podUID="37815aee-acec-464a-bb97-33f9318dbc8a"
Jul 14 20:15:14.330323 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.330283    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"]
Jul 14 20:15:14.330945 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.330750    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7555b08-4c25-4de4-b270-6dcab0971c05" containerName="cleanup"
Jul 14 20:15:14.330945 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.330761    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7555b08-4c25-4de4-b270-6dcab0971c05" containerName="cleanup"
Jul 14 20:15:14.330945 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.330828    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7555b08-4c25-4de4-b270-6dcab0971c05" containerName="cleanup"
Jul 14 20:15:14.334415 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.334373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.337330 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.337130    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6023fdbc\"/\"openshift-service-ca.crt\""
Jul 14 20:15:14.337330 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.337201    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6023fdbc\"/\"kube-root-ca.crt\""
Jul 14 20:15:14.338270 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.338082    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6023fdbc\"/\"default-dockercfg-vbk7p\""
Jul 14 20:15:14.338270 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.338159    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6023fdbc\"/\"e2e-lifecycle-model-6023fdbc-kserve-self-signed-certs\""
Jul 14 20:15:14.350525 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.350488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"]
Jul 14 20:15:14.488693 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.488657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-dshm\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.488909 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.488720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tmp-dir\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.488909 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.488818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-model-cache\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.488909 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.488838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-home\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.488909 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.488857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlv8p\" (UniqueName: \"kubernetes.io/projected/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-kube-api-access-zlv8p\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.488909 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.488897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tls-certs\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.589977 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.589863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-model-cache\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.589977 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.589912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-home\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.590228 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.590037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlv8p\" (UniqueName: \"kubernetes.io/projected/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-kube-api-access-zlv8p\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.590228 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.590169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tls-certs\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.590228 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.590221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-dshm\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.590412 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.590267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-home\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.590412 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.590281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tmp-dir\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.590412 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.590321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-model-cache\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.590561 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.590544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tmp-dir\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.592560 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.592537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-dshm\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.592813 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.592795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tls-certs\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.598384 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.598358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlv8p\" (UniqueName: \"kubernetes.io/projected/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-kube-api-access-zlv8p\") pod \"e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") " pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.645502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.645444    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:14.799704 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:14.799672    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"]
Jul 14 20:15:14.801891 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:15:14.801843    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7e7b494_4a9f_4eec_8a16_c4b40e73481d.slice/crio-0dba49604f263145cf6c22b89ab75243964685adfba7350e4b7aacca34d7324e WatchSource:0}: Error finding container 0dba49604f263145cf6c22b89ab75243964685adfba7350e4b7aacca34d7324e: Status 404 returned error can't find the container with id 0dba49604f263145cf6c22b89ab75243964685adfba7350e4b7aacca34d7324e
Jul 14 20:15:15.006781 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:15.006739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr" event={"ID":"b7e7b494-4a9f-4eec-8a16-c4b40e73481d","Type":"ContainerStarted","Data":"0dba49604f263145cf6c22b89ab75243964685adfba7350e4b7aacca34d7324e"}
Jul 14 20:15:17.024565 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:17.024522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr" event={"ID":"b7e7b494-4a9f-4eec-8a16-c4b40e73481d","Type":"ContainerStarted","Data":"906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029"}
Jul 14 20:15:17.048578 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:17.048512    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr" podStartSLOduration=1.8388830010000001 podStartE2EDuration="3.048490373s" totalImagesPullingTime="1.209607372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:15:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:15:17.046265424 +0000 UTC m=+2215.357964896" watchObservedRunningTime="2026-07-14 20:15:17.048490373 +0000 UTC m=+2215.360189829"
Jul 14 20:15:18.332589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:18.332552    2581 scope.go:117] "RemoveContainer" containerID="6f439a20973e1461cece0d48fad5d489a4d46a18a70b0c37122e0546e3e16577"
Jul 14 20:15:18.528631 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:18.528591    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:15:19.040139 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:19.040045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" event={"ID":"37815aee-acec-464a-bb97-33f9318dbc8a","Type":"ContainerStarted","Data":"0ec94f0bd4987758305318eef870cfec23a8f2e8176a6ae0f5246f33de8f3dcc"}
Jul 14 20:15:20.048199 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:20.048162    2581 generic.go:356] "Generic (PLEG): container finished" podID="37815aee-acec-464a-bb97-33f9318dbc8a" containerID="0ec94f0bd4987758305318eef870cfec23a8f2e8176a6ae0f5246f33de8f3dcc" exitCode=0
Jul 14 20:15:20.048671 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:20.048234    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" event={"ID":"37815aee-acec-464a-bb97-33f9318dbc8a","Type":"ContainerDied","Data":"0ec94f0bd4987758305318eef870cfec23a8f2e8176a6ae0f5246f33de8f3dcc"}
Jul 14 20:15:20.048671 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:20.048306    2581 scope.go:117] "RemoveContainer" containerID="6f439a20973e1461cece0d48fad5d489a4d46a18a70b0c37122e0546e3e16577"
Jul 14 20:15:20.069750 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:20.069719    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"]
Jul 14 20:15:21.208216 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:21.208191    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"
Jul 14 20:15:22.064599 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:22.064569    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"
Jul 14 20:15:22.064599 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:22.064578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8" event={"ID":"37815aee-acec-464a-bb97-33f9318dbc8a","Type":"ContainerDied","Data":"cbd507138761d56acd4528e282015ba359e2bd06469595a5aa21bc5de3e8c44d"}
Jul 14 20:15:22.064868 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:22.064629    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbd507138761d56acd4528e282015ba359e2bd06469595a5aa21bc5de3e8c44d"
Jul 14 20:15:22.104178 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:22.104141    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"]
Jul 14 20:15:22.108314 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:22.108285    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6023fdbc-29734335-7kdd8"]
Jul 14 20:15:23.338830 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:23.338797    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" path="/var/lib/kubelet/pods/37815aee-acec-464a-bb97-33f9318dbc8a/volumes"
Jul 14 20:15:23.674147 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:23.674050    2581 scope.go:117] "RemoveContainer" containerID="ae330abd78b5f50f0f2e3ff1b20f6aca1befccf197c1c102229f4ee22a0f5bfa"
Jul 14 20:15:24.646271 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:24.646227    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:24.646734 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:24.646285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:24.659457 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:24.659428    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:25.096913 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:25.096882    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:15:39.534797 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:39.534762    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:15:45.310764 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.310664    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"]
Jul 14 20:15:45.311238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.311121    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:15:45.311238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.311134    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:15:45.311238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.311143    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:15:45.311238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.311149    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:15:45.311238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.311227    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:15:45.311238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.311240    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:15:45.314733 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.314711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"
Jul 14 20:15:45.318273 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.318247    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:15:45.318776 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.318752    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:15:45.327082 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.327048    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"]
Jul 14 20:15:45.389032 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.388992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm\" (UID: \"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"
Jul 14 20:15:45.489934 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.489892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm\" (UID: \"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"
Jul 14 20:15:45.490677 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.490649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm\" (UID: \"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"
Jul 14 20:15:45.627009 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.626896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"
Jul 14 20:15:45.777502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:45.777471    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"]
Jul 14 20:15:45.779631 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:15:45.779595    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cc34a9_ea2b_4ce2_a1f6_37d88fbf90bf.slice/crio-736d609807845f9b0cfc7e2807436ef663df9fab892ea8b8bd8b307050d0ef57 WatchSource:0}: Error finding container 736d609807845f9b0cfc7e2807436ef663df9fab892ea8b8bd8b307050d0ef57: Status 404 returned error can't find the container with id 736d609807845f9b0cfc7e2807436ef663df9fab892ea8b8bd8b307050d0ef57
Jul 14 20:15:46.231155 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:46.231055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm" event={"ID":"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf","Type":"ContainerStarted","Data":"9627e193a2cd148b1de4dbdd74e48d28bc5936efd49c618d05a450aaf3918565"}
Jul 14 20:15:46.231155 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:46.231115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm" event={"ID":"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf","Type":"ContainerStarted","Data":"736d609807845f9b0cfc7e2807436ef663df9fab892ea8b8bd8b307050d0ef57"}
Jul 14 20:15:46.246796 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:46.246730    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm" podStartSLOduration=1.246709059 podStartE2EDuration="1.246709059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:15:46.245695566 +0000 UTC m=+2244.557395016" watchObservedRunningTime="2026-07-14 20:15:46.246709059 +0000 UTC m=+2244.558408519"
Jul 14 20:15:47.239391 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:47.239351    2581 generic.go:356] "Generic (PLEG): container finished" podID="89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf" containerID="9627e193a2cd148b1de4dbdd74e48d28bc5936efd49c618d05a450aaf3918565" exitCode=0
Jul 14 20:15:47.239826 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:47.239439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm" event={"ID":"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf","Type":"ContainerDied","Data":"9627e193a2cd148b1de4dbdd74e48d28bc5936efd49c618d05a450aaf3918565"}
Jul 14 20:15:48.402104 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:48.402077    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"
Jul 14 20:15:48.523020 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:48.522914    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf-maas-api-service-ca\") pod \"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf\" (UID: \"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf\") "
Jul 14 20:15:48.523323 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:48.523303    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf" (UID: "89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:15:48.623828 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:48.623793    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:15:49.253378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:49.253346    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm"
Jul 14 20:15:49.253578 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:49.253386    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm" event={"ID":"89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf","Type":"ContainerDied","Data":"736d609807845f9b0cfc7e2807436ef663df9fab892ea8b8bd8b307050d0ef57"}
Jul 14 20:15:49.253578 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:49.253451    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="736d609807845f9b0cfc7e2807436ef663df9fab892ea8b8bd8b307050d0ef57"
Jul 14 20:15:59.459434 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:15:59.458027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:16:05.418935 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:05.418893    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"]
Jul 14 20:16:20.440210 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.440147    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr" podUID="b7e7b494-4a9f-4eec-8a16-c4b40e73481d" containerName="main" containerID="cri-o://906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029" gracePeriod=45
Jul 14 20:16:20.714597 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.714567    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:16:20.790672 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.790640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-model-cache\") pod \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") "
Jul 14 20:16:20.790891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.790688    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-home\") pod \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") "
Jul 14 20:16:20.790891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.790747    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-dshm\") pod \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") "
Jul 14 20:16:20.790891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.790788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tmp-dir\") pod \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") "
Jul 14 20:16:20.790891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.790819    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlv8p\" (UniqueName: \"kubernetes.io/projected/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-kube-api-access-zlv8p\") pod \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") "
Jul 14 20:16:20.791122 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.790921    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tls-certs\") pod \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\" (UID: \"b7e7b494-4a9f-4eec-8a16-c4b40e73481d\") "
Jul 14 20:16:20.791122 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.790963    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7e7b494-4a9f-4eec-8a16-c4b40e73481d" (UID: "b7e7b494-4a9f-4eec-8a16-c4b40e73481d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:16:20.791122 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.791023    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-home" (OuterVolumeSpecName: "home") pod "b7e7b494-4a9f-4eec-8a16-c4b40e73481d" (UID: "b7e7b494-4a9f-4eec-8a16-c4b40e73481d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:16:20.791122 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.791088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7e7b494-4a9f-4eec-8a16-c4b40e73481d" (UID: "b7e7b494-4a9f-4eec-8a16-c4b40e73481d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:16:20.791292 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.791274    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-model-cache\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:16:20.791329 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.791298    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-home\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:16:20.791329 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.791311    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tmp-dir\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:16:20.793131 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.793105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-dshm" (OuterVolumeSpecName: "dshm") pod "b7e7b494-4a9f-4eec-8a16-c4b40e73481d" (UID: "b7e7b494-4a9f-4eec-8a16-c4b40e73481d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:16:20.793131 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.793113    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7e7b494-4a9f-4eec-8a16-c4b40e73481d" (UID: "b7e7b494-4a9f-4eec-8a16-c4b40e73481d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:16:20.793291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.793193    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-kube-api-access-zlv8p" (OuterVolumeSpecName: "kube-api-access-zlv8p") pod "b7e7b494-4a9f-4eec-8a16-c4b40e73481d" (UID: "b7e7b494-4a9f-4eec-8a16-c4b40e73481d"). InnerVolumeSpecName "kube-api-access-zlv8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:16:20.892490 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.892451    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-dshm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:16:20.892490 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.892484    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlv8p\" (UniqueName: \"kubernetes.io/projected/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-kube-api-access-zlv8p\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:16:20.892490 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:20.892498    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7b494-4a9f-4eec-8a16-c4b40e73481d-tls-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:16:21.470022 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.469986    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7e7b494-4a9f-4eec-8a16-c4b40e73481d" containerID="906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029" exitCode=0
Jul 14 20:16:21.470499 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.470037    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"
Jul 14 20:16:21.470499 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.470072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr" event={"ID":"b7e7b494-4a9f-4eec-8a16-c4b40e73481d","Type":"ContainerDied","Data":"906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029"}
Jul 14 20:16:21.470499 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.470116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr" event={"ID":"b7e7b494-4a9f-4eec-8a16-c4b40e73481d","Type":"ContainerDied","Data":"0dba49604f263145cf6c22b89ab75243964685adfba7350e4b7aacca34d7324e"}
Jul 14 20:16:21.470499 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.470136    2581 scope.go:117] "RemoveContainer" containerID="906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029"
Jul 14 20:16:21.480019 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.480000    2581 scope.go:117] "RemoveContainer" containerID="906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029"
Jul 14 20:16:21.480298 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:16:21.480278    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029\": container with ID starting with 906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029 not found: ID does not exist" containerID="906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029"
Jul 14 20:16:21.480382 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.480306    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029"} err="failed to get container status \"906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029\": rpc error: code = NotFound desc = could not find container \"906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029\": container with ID starting with 906cfb70832950367ec0f0cdfcbf7b7936293ed047c154076fceaf9ce3876029 not found: ID does not exist"
Jul 14 20:16:21.490622 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.490589    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"]
Jul 14 20:16:21.496030 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:21.496004    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6023fdbc/e2e-lifecycle-model-6023fdbc-kserve-66bccc7d99-29dqr"]
Jul 14 20:16:23.345303 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:23.345121    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7e7b494-4a9f-4eec-8a16-c4b40e73481d" path="/var/lib/kubelet/pods/b7e7b494-4a9f-4eec-8a16-c4b40e73481d/volumes"
Jul 14 20:16:23.698742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:23.698717    2581 scope.go:117] "RemoveContainer" containerID="e60cf022a68bc3a3228c4c6a15fdc4355ac5cc10f0343cbae9908029bf9157e8"
Jul 14 20:16:57.531464 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:16:57.531424    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:17:12.735372 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:17:12.735330    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:17:23.727978 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:17:23.727892    2581 scope.go:117] "RemoveContainer" containerID="0f3450558d593373d2f484481e8624809af7773029863ad88b7c593cb51c109f"
Jul 14 20:17:56.338355 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:17:56.338303    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:18:00.214264 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.214216    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"]
Jul 14 20:18:00.214963 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.214937    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:18:00.215039 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.214967    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:18:00.215039 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.214988    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf" containerName="revoke-keys"
Jul 14 20:18:00.215039 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.214997    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf" containerName="revoke-keys"
Jul 14 20:18:00.215039 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.215011    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e7b494-4a9f-4eec-8a16-c4b40e73481d" containerName="main"
Jul 14 20:18:00.215039 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.215021    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e7b494-4a9f-4eec-8a16-c4b40e73481d" containerName="main"
Jul 14 20:18:00.215303 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.215132    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf" containerName="revoke-keys"
Jul 14 20:18:00.215303 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.215145    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="37815aee-acec-464a-bb97-33f9318dbc8a" containerName="cleanup"
Jul 14 20:18:00.215303 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.215157    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e7b494-4a9f-4eec-8a16-c4b40e73481d" containerName="main"
Jul 14 20:18:00.219119 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.219084    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.222543 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.222495    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:18:00.222543 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.222533    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 20:18:00.222793 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.222647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3195eb-serving-cert\""
Jul 14 20:18:00.223359 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.223341    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tfc6h\""
Jul 14 20:18:00.229861 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.229834    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"]
Jul 14 20:18:00.333230 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.333186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b3059f0-ed95-47b0-a136-1a74c28c9834-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.333485 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.333288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b3059f0-ed95-47b0-a136-1a74c28c9834-maas-api-tls\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.333485 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.333348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8ktg\" (UniqueName: \"kubernetes.io/projected/9b3059f0-ed95-47b0-a136-1a74c28c9834-kube-api-access-h8ktg\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.434189 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.434152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b3059f0-ed95-47b0-a136-1a74c28c9834-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.434460 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.434207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b3059f0-ed95-47b0-a136-1a74c28c9834-maas-api-tls\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.434460 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.434249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8ktg\" (UniqueName: \"kubernetes.io/projected/9b3059f0-ed95-47b0-a136-1a74c28c9834-kube-api-access-h8ktg\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.434939 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.434913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b3059f0-ed95-47b0-a136-1a74c28c9834-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.436763 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.436741    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b3059f0-ed95-47b0-a136-1a74c28c9834-maas-api-tls\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.443543 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.443518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8ktg\" (UniqueName: \"kubernetes.io/projected/9b3059f0-ed95-47b0-a136-1a74c28c9834-kube-api-access-h8ktg\") pod \"maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.533824 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.533723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:00.692592 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.692523    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"]
Jul 14 20:18:00.694216 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:18:00.694173    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3059f0_ed95_47b0_a136_1a74c28c9834.slice/crio-487fd7cccfaa685b0168dde9160bd1ae5a6b50835cc0392df254660dcfdbf679 WatchSource:0}: Error finding container 487fd7cccfaa685b0168dde9160bd1ae5a6b50835cc0392df254660dcfdbf679: Status 404 returned error can't find the container with id 487fd7cccfaa685b0168dde9160bd1ae5a6b50835cc0392df254660dcfdbf679
Jul 14 20:18:00.695839 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:00.695818    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 20:18:01.106964 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:01.106923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn" event={"ID":"9b3059f0-ed95-47b0-a136-1a74c28c9834","Type":"ContainerStarted","Data":"487fd7cccfaa685b0168dde9160bd1ae5a6b50835cc0392df254660dcfdbf679"}
Jul 14 20:18:02.114803 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:02.114710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn" event={"ID":"9b3059f0-ed95-47b0-a136-1a74c28c9834","Type":"ContainerStarted","Data":"43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e"}
Jul 14 20:18:02.115228 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:02.114833    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:02.132636 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:02.132579    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn" podStartSLOduration=1.115208679 podStartE2EDuration="2.132561148s" totalImagesPullingTime="1.017352469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:18:02.131438361 +0000 UTC m=+2380.443137811" watchObservedRunningTime="2026-07-14 20:18:02.132561148 +0000 UTC m=+2380.444260603"
Jul 14 20:18:08.130416 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:08.130381    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:18:11.930295 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:11.930259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:18:23.464802 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:23.464772    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:18:23.476103 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:23.476077    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:18:25.632916 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.632876    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"]
Jul 14 20:18:25.637876 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.637855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.640512 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.640489    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a8298f\"/\"test-model-a8298f-kserve-self-signed-certs\""
Jul 14 20:18:25.641358 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.641328    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a8298f\"/\"kube-root-ca.crt\""
Jul 14 20:18:25.641505 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.641454    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a8298f\"/\"default-dockercfg-p2cb6\""
Jul 14 20:18:25.641557 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.641508    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a8298f\"/\"openshift-service-ca.crt\""
Jul 14 20:18:25.649325 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.649299    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"]
Jul 14 20:18:25.783378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.783326    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-model-cache\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.783378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.783380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-dshm\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.783643 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.783469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-home\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.783643 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.783493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg2sm\" (UniqueName: \"kubernetes.io/projected/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-kube-api-access-jg2sm\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.783643 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.783617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tls-certs\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.783765 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.783687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tmp-dir\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885135 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tls-certs\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885135 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tmp-dir\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885376 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-model-cache\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885376 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-dshm\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885376 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885213    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-home\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885376 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg2sm\" (UniqueName: \"kubernetes.io/projected/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-kube-api-access-jg2sm\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885643 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885570    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tmp-dir\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885643 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-model-cache\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.885858 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.885832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-home\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.887754 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.887719    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tls-certs\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.887938 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.887915    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-dshm\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.893568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.893504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg2sm\" (UniqueName: \"kubernetes.io/projected/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-kube-api-access-jg2sm\") pod \"test-model-a8298f-kserve-7d7997489-mdlwg\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") " pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:25.948747 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:25.948703    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:26.109610 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:26.109579    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"]
Jul 14 20:18:26.111344 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:18:26.111314    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0d45a4a_a13b_4a7c_8146_ee47d8a4fbdc.slice/crio-476697ce32a7f739319c196e59f620648e316108aef64d26cd56e5aea49c1aff WatchSource:0}: Error finding container 476697ce32a7f739319c196e59f620648e316108aef64d26cd56e5aea49c1aff: Status 404 returned error can't find the container with id 476697ce32a7f739319c196e59f620648e316108aef64d26cd56e5aea49c1aff
Jul 14 20:18:26.275531 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:26.275494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg" event={"ID":"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc","Type":"ContainerStarted","Data":"fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243"}
Jul 14 20:18:26.275709 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:26.275541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg" event={"ID":"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc","Type":"ContainerStarted","Data":"476697ce32a7f739319c196e59f620648e316108aef64d26cd56e5aea49c1aff"}
Jul 14 20:18:26.293788 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:26.293729    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg" podStartSLOduration=1.293710109 podStartE2EDuration="1.293710109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:18:26.291299817 +0000 UTC m=+2404.602999273" watchObservedRunningTime="2026-07-14 20:18:26.293710109 +0000 UTC m=+2404.605409597"
Jul 14 20:18:29.323458 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:29.323420    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:18:35.949181 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:35.949139    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:35.949811 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:35.949285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:35.962185 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:35.962153    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:36.351190 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:36.351100    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:18:43.626083 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:43.625992    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:18:45.930304 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.930263    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"]
Jul 14 20:18:45.934371 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.934345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:45.936597 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.936572    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3195eb\"/\"openshift-service-ca.crt\""
Jul 14 20:18:45.937458 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.937354    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3195eb\"/\"default-dockercfg-ptvgg\""
Jul 14 20:18:45.937576 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.937394    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3195eb\"/\"kube-root-ca.crt\""
Jul 14 20:18:45.937634 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.937354    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3195eb\"/\"test-model-3195eb-kserve-self-signed-certs\""
Jul 14 20:18:45.941437 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.941392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"]
Jul 14 20:18:45.969504 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.969467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-model-cache\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:45.969729 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.969520    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8vzq\" (UniqueName: \"kubernetes.io/projected/e73bbf10-f055-41c8-b188-2bb964cae402-kube-api-access-w8vzq\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:45.969729 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.969545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-home\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:45.969729 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.969706    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-dshm\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:45.969860 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.969774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e73bbf10-f055-41c8-b188-2bb964cae402-tls-certs\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:45.969860 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:45.969806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-tmp-dir\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071179 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e73bbf10-f055-41c8-b188-2bb964cae402-tls-certs\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071179 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-tmp-dir\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-model-cache\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8vzq\" (UniqueName: \"kubernetes.io/projected/e73bbf10-f055-41c8-b188-2bb964cae402-kube-api-access-w8vzq\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071502 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-home\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071658 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071543    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-dshm\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071658 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-tmp-dir\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071768 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071675    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-model-cache\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.071883 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.071859    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-home\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.073994 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.073968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-dshm\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.074361 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.074339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e73bbf10-f055-41c8-b188-2bb964cae402-tls-certs\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.079914 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.079889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8vzq\" (UniqueName: \"kubernetes.io/projected/e73bbf10-f055-41c8-b188-2bb964cae402-kube-api-access-w8vzq\") pod \"test-model-3195eb-kserve-574b99f7c9-zbhm2\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") " pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.247853 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.247815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:46.402890 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:46.402857    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"]
Jul 14 20:18:46.404709 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:18:46.404680    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode73bbf10_f055_41c8_b188_2bb964cae402.slice/crio-3f867aa5177e73846f7c997245cf63fb3d099be416facdea2f719f0216ad6634 WatchSource:0}: Error finding container 3f867aa5177e73846f7c997245cf63fb3d099be416facdea2f719f0216ad6634: Status 404 returned error can't find the container with id 3f867aa5177e73846f7c997245cf63fb3d099be416facdea2f719f0216ad6634
Jul 14 20:18:47.408622 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:47.408580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2" event={"ID":"e73bbf10-f055-41c8-b188-2bb964cae402","Type":"ContainerStarted","Data":"91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975"}
Jul 14 20:18:47.408622 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:47.408630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2" event={"ID":"e73bbf10-f055-41c8-b188-2bb964cae402","Type":"ContainerStarted","Data":"3f867aa5177e73846f7c997245cf63fb3d099be416facdea2f719f0216ad6634"}
Jul 14 20:18:47.427841 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:47.427788    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2" podStartSLOduration=2.427770299 podStartE2EDuration="2.427770299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:18:47.424943329 +0000 UTC m=+2425.736642830" watchObservedRunningTime="2026-07-14 20:18:47.427770299 +0000 UTC m=+2425.739469763"
Jul 14 20:18:56.248933 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:56.248891    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:56.249346 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:56.248955    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:56.261393 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:56.261355    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:18:56.479081 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:18:56.479052    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:19:02.233633 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:02.233593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:19:08.532711 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:08.532663    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"]
Jul 14 20:19:09.581097 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.581059    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"]
Jul 14 20:19:09.691741 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.691704    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"]
Jul 14 20:19:09.695652 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.695629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"
Jul 14 20:19:09.698078 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.698056    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:19:09.705628 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.704472    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"]
Jul 14 20:19:09.806358 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.806313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07e3d2b5-ed17-414a-8e3d-ea269ba95e4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s\" (UID: \"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"
Jul 14 20:19:09.907940 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.907834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07e3d2b5-ed17-414a-8e3d-ea269ba95e4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s\" (UID: \"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"
Jul 14 20:19:09.908582 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:09.908557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07e3d2b5-ed17-414a-8e3d-ea269ba95e4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s\" (UID: \"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"
Jul 14 20:19:10.011057 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:10.011008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"
Jul 14 20:19:10.176124 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:10.176091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"]
Jul 14 20:19:10.179541 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:19:10.179511    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e3d2b5_ed17_414a_8e3d_ea269ba95e4c.slice/crio-b552e52fb6383036a1f97968f412243fb462024eed92ffa676868f0c39ac1d5c WatchSource:0}: Error finding container b552e52fb6383036a1f97968f412243fb462024eed92ffa676868f0c39ac1d5c: Status 404 returned error can't find the container with id b552e52fb6383036a1f97968f412243fb462024eed92ffa676868f0c39ac1d5c
Jul 14 20:19:10.567287 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:10.567230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s" event={"ID":"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c","Type":"ContainerStarted","Data":"55639a327ef064bf9ce35de6405a14504af7efb16a6127d3c072210d6ee73860"}
Jul 14 20:19:10.567520 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:10.567317    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s" event={"ID":"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c","Type":"ContainerStarted","Data":"b552e52fb6383036a1f97968f412243fb462024eed92ffa676868f0c39ac1d5c"}
Jul 14 20:19:10.584154 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:10.584102    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s" podStartSLOduration=1.584085419 podStartE2EDuration="1.584085419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:19:10.581456923 +0000 UTC m=+2448.893156382" watchObservedRunningTime="2026-07-14 20:19:10.584085419 +0000 UTC m=+2448.895784906"
Jul 14 20:19:11.575214 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:11.575174    2581 generic.go:356] "Generic (PLEG): container finished" podID="07e3d2b5-ed17-414a-8e3d-ea269ba95e4c" containerID="55639a327ef064bf9ce35de6405a14504af7efb16a6127d3c072210d6ee73860" exitCode=0
Jul 14 20:19:11.575459 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:11.575264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s" event={"ID":"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c","Type":"ContainerDied","Data":"55639a327ef064bf9ce35de6405a14504af7efb16a6127d3c072210d6ee73860"}
Jul 14 20:19:12.735187 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:12.735156    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"
Jul 14 20:19:12.836877 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:12.836775    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07e3d2b5-ed17-414a-8e3d-ea269ba95e4c-maas-api-service-ca\") pod \"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c\" (UID: \"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c\") "
Jul 14 20:19:12.837168 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:12.837142    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07e3d2b5-ed17-414a-8e3d-ea269ba95e4c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "07e3d2b5-ed17-414a-8e3d-ea269ba95e4c" (UID: "07e3d2b5-ed17-414a-8e3d-ea269ba95e4c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:19:12.938241 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:12.938198    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07e3d2b5-ed17-414a-8e3d-ea269ba95e4c-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:13.589076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:13.589046    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s"
Jul 14 20:19:13.589267 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:13.589089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s" event={"ID":"07e3d2b5-ed17-414a-8e3d-ea269ba95e4c","Type":"ContainerDied","Data":"b552e52fb6383036a1f97968f412243fb462024eed92ffa676868f0c39ac1d5c"}
Jul 14 20:19:13.589267 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:13.589143    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b552e52fb6383036a1f97968f412243fb462024eed92ffa676868f0c39ac1d5c"
Jul 14 20:19:17.839047 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:17.839005    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:19:23.554834 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.554785    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg" podUID="b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" containerName="main" containerID="cri-o://fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243" gracePeriod=45
Jul 14 20:19:23.825663 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.825635    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:19:23.843416 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.843353    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-dshm\") pod \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") "
Jul 14 20:19:23.843416 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.843429    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-model-cache\") pod \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") "
Jul 14 20:19:23.843879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.843478    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tls-certs\") pod \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") "
Jul 14 20:19:23.843879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.843543    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-home\") pod \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") "
Jul 14 20:19:23.843879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.843616    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tmp-dir\") pod \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") "
Jul 14 20:19:23.843879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.843790    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg2sm\" (UniqueName: \"kubernetes.io/projected/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-kube-api-access-jg2sm\") pod \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\" (UID: \"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc\") "
Jul 14 20:19:23.844229 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.844158    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" (UID: "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:23.844229 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.844204    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-home" (OuterVolumeSpecName: "home") pod "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" (UID: "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:23.844445 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.844354    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" (UID: "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:23.846216 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.846179    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-dshm" (OuterVolumeSpecName: "dshm") pod "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" (UID: "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:23.846355 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.846274    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" (UID: "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:19:23.846583 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.846559    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-kube-api-access-jg2sm" (OuterVolumeSpecName: "kube-api-access-jg2sm") pod "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" (UID: "b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc"). InnerVolumeSpecName "kube-api-access-jg2sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:19:23.945375 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.945335    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg2sm\" (UniqueName: \"kubernetes.io/projected/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-kube-api-access-jg2sm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:23.945375 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.945371    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-dshm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:23.945375 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.945382    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-model-cache\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:23.945658 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.945393    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tls-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:23.945658 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.945419    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-home\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:23.945658 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:23.945432    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc-tmp-dir\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:24.604112 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.604075    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2" podUID="e73bbf10-f055-41c8-b188-2bb964cae402" containerName="main" containerID="cri-o://91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975" gracePeriod=45
Jul 14 20:19:24.666598 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.666561    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" containerID="fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243" exitCode=0
Jul 14 20:19:24.666810 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.666629    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"
Jul 14 20:19:24.666810 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.666654    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg" event={"ID":"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc","Type":"ContainerDied","Data":"fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243"}
Jul 14 20:19:24.666810 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.666707    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg" event={"ID":"b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc","Type":"ContainerDied","Data":"476697ce32a7f739319c196e59f620648e316108aef64d26cd56e5aea49c1aff"}
Jul 14 20:19:24.666810 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.666728    2581 scope.go:117] "RemoveContainer" containerID="fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243"
Jul 14 20:19:24.692247 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.692215    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"]
Jul 14 20:19:24.697769 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.697736    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a8298f/test-model-a8298f-kserve-7d7997489-mdlwg"]
Jul 14 20:19:24.718588 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.718560    2581 scope.go:117] "RemoveContainer" containerID="fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243"
Jul 14 20:19:24.718969 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:19:24.718944    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243\": container with ID starting with fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243 not found: ID does not exist" containerID="fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243"
Jul 14 20:19:24.719053 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.718988    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243"} err="failed to get container status \"fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243\": rpc error: code = NotFound desc = could not find container \"fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243\": container with ID starting with fda24badb605424a45f54536d139bec6836c9b3de331d526a38fad6c0f162243 not found: ID does not exist"
Jul 14 20:19:24.876790 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.876759    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:19:24.954645 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-dshm\") pod \"e73bbf10-f055-41c8-b188-2bb964cae402\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") "
Jul 14 20:19:24.954645 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954663    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-model-cache\") pod \"e73bbf10-f055-41c8-b188-2bb964cae402\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") "
Jul 14 20:19:24.954879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954684    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-tmp-dir\") pod \"e73bbf10-f055-41c8-b188-2bb964cae402\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") "
Jul 14 20:19:24.954879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954770    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8vzq\" (UniqueName: \"kubernetes.io/projected/e73bbf10-f055-41c8-b188-2bb964cae402-kube-api-access-w8vzq\") pod \"e73bbf10-f055-41c8-b188-2bb964cae402\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") "
Jul 14 20:19:24.954879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-home\") pod \"e73bbf10-f055-41c8-b188-2bb964cae402\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") "
Jul 14 20:19:24.954879 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954835    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e73bbf10-f055-41c8-b188-2bb964cae402-tls-certs\") pod \"e73bbf10-f055-41c8-b188-2bb964cae402\" (UID: \"e73bbf10-f055-41c8-b188-2bb964cae402\") "
Jul 14 20:19:24.955073 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-model-cache" (OuterVolumeSpecName: "model-cache") pod "e73bbf10-f055-41c8-b188-2bb964cae402" (UID: "e73bbf10-f055-41c8-b188-2bb964cae402"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:24.955073 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.954959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e73bbf10-f055-41c8-b188-2bb964cae402" (UID: "e73bbf10-f055-41c8-b188-2bb964cae402"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:24.955196 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.955092    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-home" (OuterVolumeSpecName: "home") pod "e73bbf10-f055-41c8-b188-2bb964cae402" (UID: "e73bbf10-f055-41c8-b188-2bb964cae402"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:24.955196 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.955105    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-model-cache\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:24.955196 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.955126    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-tmp-dir\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:24.957021 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.956997    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-dshm" (OuterVolumeSpecName: "dshm") pod "e73bbf10-f055-41c8-b188-2bb964cae402" (UID: "e73bbf10-f055-41c8-b188-2bb964cae402"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:19:24.957151 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.957072    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73bbf10-f055-41c8-b188-2bb964cae402-kube-api-access-w8vzq" (OuterVolumeSpecName: "kube-api-access-w8vzq") pod "e73bbf10-f055-41c8-b188-2bb964cae402" (UID: "e73bbf10-f055-41c8-b188-2bb964cae402"). InnerVolumeSpecName "kube-api-access-w8vzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:19:24.957151 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:24.957102    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e73bbf10-f055-41c8-b188-2bb964cae402-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e73bbf10-f055-41c8-b188-2bb964cae402" (UID: "e73bbf10-f055-41c8-b188-2bb964cae402"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:19:25.056333 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.056297    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8vzq\" (UniqueName: \"kubernetes.io/projected/e73bbf10-f055-41c8-b188-2bb964cae402-kube-api-access-w8vzq\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:25.056333 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.056331    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-home\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:25.056584 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.056351    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e73bbf10-f055-41c8-b188-2bb964cae402-tls-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:25.056584 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.056363    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e73bbf10-f055-41c8-b188-2bb964cae402-dshm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:25.347848 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.347809    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" path="/var/lib/kubelet/pods/b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc/volumes"
Jul 14 20:19:25.678742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.678644    2581 generic.go:356] "Generic (PLEG): container finished" podID="e73bbf10-f055-41c8-b188-2bb964cae402" containerID="91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975" exitCode=0
Jul 14 20:19:25.678742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.678705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2" event={"ID":"e73bbf10-f055-41c8-b188-2bb964cae402","Type":"ContainerDied","Data":"91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975"}
Jul 14 20:19:25.678742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.678715    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"
Jul 14 20:19:25.678742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.678739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2" event={"ID":"e73bbf10-f055-41c8-b188-2bb964cae402","Type":"ContainerDied","Data":"3f867aa5177e73846f7c997245cf63fb3d099be416facdea2f719f0216ad6634"}
Jul 14 20:19:25.679612 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.678759    2581 scope.go:117] "RemoveContainer" containerID="91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975"
Jul 14 20:19:25.689369 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.689340    2581 scope.go:117] "RemoveContainer" containerID="91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975"
Jul 14 20:19:25.689735 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:19:25.689712    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975\": container with ID starting with 91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975 not found: ID does not exist" containerID="91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975"
Jul 14 20:19:25.689812 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.689749    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975"} err="failed to get container status \"91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975\": rpc error: code = NotFound desc = could not find container \"91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975\": container with ID starting with 91e22dd61d3b92ec48b145bf36c80ee2ccd702063a17f65fca532d38b4fc1975 not found: ID does not exist"
Jul 14 20:19:25.707085 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.707044    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"]
Jul 14 20:19:25.711599 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:25.711565    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3195eb/test-model-3195eb-kserve-574b99f7c9-zbhm2"]
Jul 14 20:19:27.339816 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:27.339779    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e73bbf10-f055-41c8-b188-2bb964cae402" path="/var/lib/kubelet/pods/e73bbf10-f055-41c8-b188-2bb964cae402/volumes"
Jul 14 20:19:31.679383 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:31.679339    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:19:35.874533 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.874492    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"]
Jul 14 20:19:35.874984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.874942    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e73bbf10-f055-41c8-b188-2bb964cae402" containerName="main"
Jul 14 20:19:35.874984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.874953    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e73bbf10-f055-41c8-b188-2bb964cae402" containerName="main"
Jul 14 20:19:35.874984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.874962    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e3d2b5-ed17-414a-8e3d-ea269ba95e4c" containerName="revoke-keys"
Jul 14 20:19:35.874984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.874968    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e3d2b5-ed17-414a-8e3d-ea269ba95e4c" containerName="revoke-keys"
Jul 14 20:19:35.874984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.874981    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" containerName="main"
Jul 14 20:19:35.874984 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.874986    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" containerName="main"
Jul 14 20:19:35.875181 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.875046    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e73bbf10-f055-41c8-b188-2bb964cae402" containerName="main"
Jul 14 20:19:35.875181 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.875056    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0d45a4a-a13b-4a7c-8146-ee47d8a4fbdc" containerName="main"
Jul 14 20:19:35.875181 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.875062    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="07e3d2b5-ed17-414a-8e3d-ea269ba95e4c" containerName="revoke-keys"
Jul 14 20:19:35.878436 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.878391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"
Jul 14 20:19:35.881060 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.881033    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:19:35.887121 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.887091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"]
Jul 14 20:19:35.958668 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:35.958634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5635d2f3-e515-4220-ae4a-7e69f8950c83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt\" (UID: \"5635d2f3-e515-4220-ae4a-7e69f8950c83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"
Jul 14 20:19:36.059826 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:36.059786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5635d2f3-e515-4220-ae4a-7e69f8950c83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt\" (UID: \"5635d2f3-e515-4220-ae4a-7e69f8950c83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"
Jul 14 20:19:36.060462 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:36.060443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5635d2f3-e515-4220-ae4a-7e69f8950c83-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt\" (UID: \"5635d2f3-e515-4220-ae4a-7e69f8950c83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"
Jul 14 20:19:36.190506 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:36.190382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"
Jul 14 20:19:36.354836 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:36.354800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"]
Jul 14 20:19:36.356270 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:19:36.356238    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5635d2f3_e515_4220_ae4a_7e69f8950c83.slice/crio-2fd31ca0e6a6e1976745624f910c239565d3f33539018e213635b686a78a97f8 WatchSource:0}: Error finding container 2fd31ca0e6a6e1976745624f910c239565d3f33539018e213635b686a78a97f8: Status 404 returned error can't find the container with id 2fd31ca0e6a6e1976745624f910c239565d3f33539018e213635b686a78a97f8
Jul 14 20:19:36.753955 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:36.753913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt" event={"ID":"5635d2f3-e515-4220-ae4a-7e69f8950c83","Type":"ContainerStarted","Data":"11d0fb6cfb652e6c03ace42beac80d0a9465622c37efeeec6e6a549136711758"}
Jul 14 20:19:36.754165 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:36.753972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt" event={"ID":"5635d2f3-e515-4220-ae4a-7e69f8950c83","Type":"ContainerStarted","Data":"2fd31ca0e6a6e1976745624f910c239565d3f33539018e213635b686a78a97f8"}
Jul 14 20:19:36.774197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:36.774138    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt" podStartSLOduration=1.774116464 podStartE2EDuration="1.774116464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:19:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:19:36.770506337 +0000 UTC m=+2475.082205806" watchObservedRunningTime="2026-07-14 20:19:36.774116464 +0000 UTC m=+2475.085815920"
Jul 14 20:19:37.761586 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:37.761548    2581 generic.go:356] "Generic (PLEG): container finished" podID="5635d2f3-e515-4220-ae4a-7e69f8950c83" containerID="11d0fb6cfb652e6c03ace42beac80d0a9465622c37efeeec6e6a549136711758" exitCode=0
Jul 14 20:19:37.762008 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:37.761626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt" event={"ID":"5635d2f3-e515-4220-ae4a-7e69f8950c83","Type":"ContainerDied","Data":"11d0fb6cfb652e6c03ace42beac80d0a9465622c37efeeec6e6a549136711758"}
Jul 14 20:19:38.916483 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:38.916453    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"
Jul 14 20:19:38.995296 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:38.995250    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5635d2f3-e515-4220-ae4a-7e69f8950c83-maas-api-service-ca\") pod \"5635d2f3-e515-4220-ae4a-7e69f8950c83\" (UID: \"5635d2f3-e515-4220-ae4a-7e69f8950c83\") "
Jul 14 20:19:38.995678 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:38.995655    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5635d2f3-e515-4220-ae4a-7e69f8950c83-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5635d2f3-e515-4220-ae4a-7e69f8950c83" (UID: "5635d2f3-e515-4220-ae4a-7e69f8950c83"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:19:39.096452 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:39.096332    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5635d2f3-e515-4220-ae4a-7e69f8950c83-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:39.775874 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:39.775842    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt"
Jul 14 20:19:39.776063 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:39.775833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt" event={"ID":"5635d2f3-e515-4220-ae4a-7e69f8950c83","Type":"ContainerDied","Data":"2fd31ca0e6a6e1976745624f910c239565d3f33539018e213635b686a78a97f8"}
Jul 14 20:19:39.776063 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:39.775970    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fd31ca0e6a6e1976745624f910c239565d3f33539018e213635b686a78a97f8"
Jul 14 20:19:46.335877 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.335839    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"]
Jul 14 20:19:46.336431 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.336125    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn" podUID="9b3059f0-ed95-47b0-a136-1a74c28c9834" containerName="maas-api" containerID="cri-o://43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e" gracePeriod=30
Jul 14 20:19:46.606310 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.606282    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:19:46.773151 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.773120    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b3059f0-ed95-47b0-a136-1a74c28c9834-maas-api-tls\") pod \"9b3059f0-ed95-47b0-a136-1a74c28c9834\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") "
Jul 14 20:19:46.773333 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.773223    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b3059f0-ed95-47b0-a136-1a74c28c9834-openshift-service-ca\") pod \"9b3059f0-ed95-47b0-a136-1a74c28c9834\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") "
Jul 14 20:19:46.773333 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.773271    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8ktg\" (UniqueName: \"kubernetes.io/projected/9b3059f0-ed95-47b0-a136-1a74c28c9834-kube-api-access-h8ktg\") pod \"9b3059f0-ed95-47b0-a136-1a74c28c9834\" (UID: \"9b3059f0-ed95-47b0-a136-1a74c28c9834\") "
Jul 14 20:19:46.773649 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.773610    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b3059f0-ed95-47b0-a136-1a74c28c9834-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b3059f0-ed95-47b0-a136-1a74c28c9834" (UID: "9b3059f0-ed95-47b0-a136-1a74c28c9834"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:19:46.775350 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.775322    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b3059f0-ed95-47b0-a136-1a74c28c9834-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b3059f0-ed95-47b0-a136-1a74c28c9834" (UID: "9b3059f0-ed95-47b0-a136-1a74c28c9834"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:19:46.775462 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.775385    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b3059f0-ed95-47b0-a136-1a74c28c9834-kube-api-access-h8ktg" (OuterVolumeSpecName: "kube-api-access-h8ktg") pod "9b3059f0-ed95-47b0-a136-1a74c28c9834" (UID: "9b3059f0-ed95-47b0-a136-1a74c28c9834"). InnerVolumeSpecName "kube-api-access-h8ktg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:19:46.823413 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.823372    2581 generic.go:356] "Generic (PLEG): container finished" podID="9b3059f0-ed95-47b0-a136-1a74c28c9834" containerID="43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e" exitCode=0
Jul 14 20:19:46.823620 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.823445    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"
Jul 14 20:19:46.823620 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.823477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn" event={"ID":"9b3059f0-ed95-47b0-a136-1a74c28c9834","Type":"ContainerDied","Data":"43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e"}
Jul 14 20:19:46.823620 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.823558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn" event={"ID":"9b3059f0-ed95-47b0-a136-1a74c28c9834","Type":"ContainerDied","Data":"487fd7cccfaa685b0168dde9160bd1ae5a6b50835cc0392df254660dcfdbf679"}
Jul 14 20:19:46.823620 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.823582    2581 scope.go:117] "RemoveContainer" containerID="43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e"
Jul 14 20:19:46.834177 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.834155    2581 scope.go:117] "RemoveContainer" containerID="43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e"
Jul 14 20:19:46.834466 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:19:46.834445    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e\": container with ID starting with 43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e not found: ID does not exist" containerID="43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e"
Jul 14 20:19:46.834529 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.834475    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e"} err="failed to get container status \"43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e\": rpc error: code = NotFound desc = could not find container \"43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e\": container with ID starting with 43f2cf546040882d2e2481316deaaf323730d9c67c9d5613cbd8f18e8eb0886e not found: ID does not exist"
Jul 14 20:19:46.846061 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.845984    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"]
Jul 14 20:19:46.849936 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.849908    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3195eb-869d96f848-dmjtn"]
Jul 14 20:19:46.874238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.874204    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b3059f0-ed95-47b0-a136-1a74c28c9834-openshift-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:46.874238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.874236    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8ktg\" (UniqueName: \"kubernetes.io/projected/9b3059f0-ed95-47b0-a136-1a74c28c9834-kube-api-access-h8ktg\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:46.874238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:46.874247    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b3059f0-ed95-47b0-a136-1a74c28c9834-maas-api-tls\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:19:47.339237 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:47.339200    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b3059f0-ed95-47b0-a136-1a74c28c9834" path="/var/lib/kubelet/pods/9b3059f0-ed95-47b0-a136-1a74c28c9834/volumes"
Jul 14 20:19:49.428236 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:19:49.428197    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:20:05.891566 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.891532    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"]
Jul 14 20:20:05.892030 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.892002    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b3059f0-ed95-47b0-a136-1a74c28c9834" containerName="maas-api"
Jul 14 20:20:05.892030 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.892017    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b3059f0-ed95-47b0-a136-1a74c28c9834" containerName="maas-api"
Jul 14 20:20:05.892110 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.892034    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5635d2f3-e515-4220-ae4a-7e69f8950c83" containerName="revoke-keys"
Jul 14 20:20:05.892110 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.892040    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5635d2f3-e515-4220-ae4a-7e69f8950c83" containerName="revoke-keys"
Jul 14 20:20:05.892179 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.892114    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b3059f0-ed95-47b0-a136-1a74c28c9834" containerName="maas-api"
Jul 14 20:20:05.892179 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.892125    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5635d2f3-e515-4220-ae4a-7e69f8950c83" containerName="revoke-keys"
Jul 14 20:20:05.895259 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.895237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:05.898024 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.897998    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tfc6h\""
Jul 14 20:20:05.898821 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.898806    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 20:20:05.898911 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.898824    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:20:05.899290 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.899267    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c73bc1-serving-cert\""
Jul 14 20:20:05.913128 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.913043    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"]
Jul 14 20:20:05.954133 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.954091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:05.954337 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.954169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p88sl\" (UniqueName: \"kubernetes.io/projected/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-kube-api-access-p88sl\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:05.954337 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:05.954268    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-openshift-service-ca\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.055578 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.055537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.055844 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.055601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p88sl\" (UniqueName: \"kubernetes.io/projected/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-kube-api-access-p88sl\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.055844 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.055649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-openshift-service-ca\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.055844 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:20:06.055715    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-serving-cert: secret "maas-api-e2e-api-a-c73bc1-serving-cert" not found
Jul 14 20:20:06.055844 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:20:06.055810    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls podName:bd4a0a38-3eeb-45fc-9474-58a9b1f234fc nodeName:}" failed. No retries permitted until 2026-07-14 20:20:06.555786746 +0000 UTC m=+2504.867486180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls") pod "maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6" (UID: "bd4a0a38-3eeb-45fc-9474-58a9b1f234fc") : secret "maas-api-e2e-api-a-c73bc1-serving-cert" not found
Jul 14 20:20:06.056536 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.056508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-openshift-service-ca\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.068191 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.068158    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p88sl\" (UniqueName: \"kubernetes.io/projected/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-kube-api-access-p88sl\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.560295 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.560258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.562863 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.562835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls\") pod \"maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.806364 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.806329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:06.969453 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:06.969421    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"]
Jul 14 20:20:06.971146 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:20:06.971116    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4a0a38_3eeb_45fc_9474_58a9b1f234fc.slice/crio-9c21106b64d3b851baae5d6bf094f2babc6baa038132405554e96a50fc6f1d20 WatchSource:0}: Error finding container 9c21106b64d3b851baae5d6bf094f2babc6baa038132405554e96a50fc6f1d20: Status 404 returned error can't find the container with id 9c21106b64d3b851baae5d6bf094f2babc6baa038132405554e96a50fc6f1d20
Jul 14 20:20:07.965380 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:07.965334    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6" event={"ID":"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc","Type":"ContainerStarted","Data":"80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592"}
Jul 14 20:20:07.965631 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:07.965443    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:07.965631 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:07.965471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6" event={"ID":"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc","Type":"ContainerStarted","Data":"9c21106b64d3b851baae5d6bf094f2babc6baa038132405554e96a50fc6f1d20"}
Jul 14 20:20:07.987280 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:07.987222    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6" podStartSLOduration=2.518979668 podStartE2EDuration="2.987200982s" totalImagesPullingTime="468.221314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:20:07.983229743 +0000 UTC m=+2506.294929214" watchObservedRunningTime="2026-07-14 20:20:07.987200982 +0000 UTC m=+2506.298900438"
Jul 14 20:20:11.253854 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:11.253811    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:20:13.979687 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:13.979606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:24.461356 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:24.461316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:20:26.563501 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.563465    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"]
Jul 14 20:20:26.569283 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.569252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"
Jul 14 20:20:26.573013 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.572990    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:20:26.582856 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.582827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"]
Jul 14 20:20:26.666194 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.666155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/296d2ea8-f18e-4e19-a0e6-6e7580f819cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f\" (UID: \"296d2ea8-f18e-4e19-a0e6-6e7580f819cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"
Jul 14 20:20:26.767934 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.767879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/296d2ea8-f18e-4e19-a0e6-6e7580f819cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f\" (UID: \"296d2ea8-f18e-4e19-a0e6-6e7580f819cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"
Jul 14 20:20:26.768556 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.768537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/296d2ea8-f18e-4e19-a0e6-6e7580f819cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f\" (UID: \"296d2ea8-f18e-4e19-a0e6-6e7580f819cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"
Jul 14 20:20:26.883789 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:26.883693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"
Jul 14 20:20:27.242488 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:27.242456    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"]
Jul 14 20:20:27.243991 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:20:27.243947    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296d2ea8_f18e_4e19_a0e6_6e7580f819cc.slice/crio-049c5dc28dbd9d6cfbdf284162546a034d267559b34979dff0c08bd0f98d1d5c WatchSource:0}: Error finding container 049c5dc28dbd9d6cfbdf284162546a034d267559b34979dff0c08bd0f98d1d5c: Status 404 returned error can't find the container with id 049c5dc28dbd9d6cfbdf284162546a034d267559b34979dff0c08bd0f98d1d5c
Jul 14 20:20:28.103730 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:28.103689    2581 generic.go:356] "Generic (PLEG): container finished" podID="296d2ea8-f18e-4e19-a0e6-6e7580f819cc" containerID="84be363d825f4c8b1d820746d4c81ec22b2ae8718f03e1fc26ac94f5bd8bb07b" exitCode=0
Jul 14 20:20:28.104130 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:28.103768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f" event={"ID":"296d2ea8-f18e-4e19-a0e6-6e7580f819cc","Type":"ContainerDied","Data":"84be363d825f4c8b1d820746d4c81ec22b2ae8718f03e1fc26ac94f5bd8bb07b"}
Jul 14 20:20:28.104130 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:28.103825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f" event={"ID":"296d2ea8-f18e-4e19-a0e6-6e7580f819cc","Type":"ContainerStarted","Data":"049c5dc28dbd9d6cfbdf284162546a034d267559b34979dff0c08bd0f98d1d5c"}
Jul 14 20:20:29.253648 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:29.253620    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"
Jul 14 20:20:29.395904 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:29.395807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/296d2ea8-f18e-4e19-a0e6-6e7580f819cc-maas-api-service-ca\") pod \"296d2ea8-f18e-4e19-a0e6-6e7580f819cc\" (UID: \"296d2ea8-f18e-4e19-a0e6-6e7580f819cc\") "
Jul 14 20:20:29.396245 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:29.396211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296d2ea8-f18e-4e19-a0e6-6e7580f819cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "296d2ea8-f18e-4e19-a0e6-6e7580f819cc" (UID: "296d2ea8-f18e-4e19-a0e6-6e7580f819cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:20:29.497052 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:29.497016    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/296d2ea8-f18e-4e19-a0e6-6e7580f819cc-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:20:30.117741 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:30.117709    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f" event={"ID":"296d2ea8-f18e-4e19-a0e6-6e7580f819cc","Type":"ContainerDied","Data":"049c5dc28dbd9d6cfbdf284162546a034d267559b34979dff0c08bd0f98d1d5c"}
Jul 14 20:20:30.117925 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:30.117762    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="049c5dc28dbd9d6cfbdf284162546a034d267559b34979dff0c08bd0f98d1d5c"
Jul 14 20:20:30.117925 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:30.117719    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f"
Jul 14 20:20:37.139958 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.139913    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"]
Jul 14 20:20:37.140637 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.140147    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6" podUID="bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" containerName="maas-api" containerID="cri-o://80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592" gracePeriod=30
Jul 14 20:20:37.405205 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.405138    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:37.475300 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.475266    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p88sl\" (UniqueName: \"kubernetes.io/projected/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-kube-api-access-p88sl\") pod \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") "
Jul 14 20:20:37.475519 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.475373    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls\") pod \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") "
Jul 14 20:20:37.475519 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.475501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-openshift-service-ca\") pod \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\" (UID: \"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc\") "
Jul 14 20:20:37.475849 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.475824    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" (UID: "bd4a0a38-3eeb-45fc-9474-58a9b1f234fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:20:37.477589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.477564    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" (UID: "bd4a0a38-3eeb-45fc-9474-58a9b1f234fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:20:37.477589 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.477578    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-kube-api-access-p88sl" (OuterVolumeSpecName: "kube-api-access-p88sl") pod "bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" (UID: "bd4a0a38-3eeb-45fc-9474-58a9b1f234fc"). InnerVolumeSpecName "kube-api-access-p88sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:20:37.577209 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.577167    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-openshift-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:20:37.577209 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.577204    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p88sl\" (UniqueName: \"kubernetes.io/projected/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-kube-api-access-p88sl\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:20:37.577209 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:37.577219    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc-maas-api-tls\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:20:38.170294 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.170259    2581 generic.go:356] "Generic (PLEG): container finished" podID="bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" containerID="80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592" exitCode=0
Jul 14 20:20:38.170766 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.170331    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"
Jul 14 20:20:38.170766 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.170352    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6" event={"ID":"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc","Type":"ContainerDied","Data":"80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592"}
Jul 14 20:20:38.170766 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.170427    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6" event={"ID":"bd4a0a38-3eeb-45fc-9474-58a9b1f234fc","Type":"ContainerDied","Data":"9c21106b64d3b851baae5d6bf094f2babc6baa038132405554e96a50fc6f1d20"}
Jul 14 20:20:38.170766 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.170452    2581 scope.go:117] "RemoveContainer" containerID="80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592"
Jul 14 20:20:38.189390 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.189370    2581 scope.go:117] "RemoveContainer" containerID="80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592"
Jul 14 20:20:38.189751 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:20:38.189728    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592\": container with ID starting with 80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592 not found: ID does not exist" containerID="80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592"
Jul 14 20:20:38.189825 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.189764    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592"} err="failed to get container status \"80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592\": rpc error: code = NotFound desc = could not find container \"80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592\": container with ID starting with 80964cea0b509c5fedc6645fe987ce3abdb0562894106b54678bf54ee4b1b592 not found: ID does not exist"
Jul 14 20:20:38.199720 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.199690    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"]
Jul 14 20:20:38.204303 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:38.204274    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c73bc1-79cc4f99dd-6xgc6"]
Jul 14 20:20:39.338857 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:39.338821    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" path="/var/lib/kubelet/pods/bd4a0a38-3eeb-45fc-9474-58a9b1f234fc/volumes"
Jul 14 20:20:40.952019 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:40.951977    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:20:52.807072 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.807032    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"]
Jul 14 20:20:52.807514 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.807500    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296d2ea8-f18e-4e19-a0e6-6e7580f819cc" containerName="revoke-keys"
Jul 14 20:20:52.807514 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.807515    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="296d2ea8-f18e-4e19-a0e6-6e7580f819cc" containerName="revoke-keys"
Jul 14 20:20:52.807593 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.807541    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" containerName="maas-api"
Jul 14 20:20:52.807593 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.807548    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" containerName="maas-api"
Jul 14 20:20:52.807659 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.807637    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="296d2ea8-f18e-4e19-a0e6-6e7580f819cc" containerName="revoke-keys"
Jul 14 20:20:52.807659 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.807647    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd4a0a38-3eeb-45fc-9474-58a9b1f234fc" containerName="maas-api"
Jul 14 20:20:52.812174 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.812152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"
Jul 14 20:20:52.814856 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.814827    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:20:52.815025 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.814893    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:20:52.817941 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.817917    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"]
Jul 14 20:20:52.931463 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:52.931381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b25e533-a235-48de-93e8-beca086ec6e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6e455f-c8r69\" (UID: \"7b25e533-a235-48de-93e8-beca086ec6e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"
Jul 14 20:20:53.032338 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:53.032287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b25e533-a235-48de-93e8-beca086ec6e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6e455f-c8r69\" (UID: \"7b25e533-a235-48de-93e8-beca086ec6e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"
Jul 14 20:20:53.033061 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:53.033033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b25e533-a235-48de-93e8-beca086ec6e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6e455f-c8r69\" (UID: \"7b25e533-a235-48de-93e8-beca086ec6e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"
Jul 14 20:20:53.125537 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:53.125433    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"
Jul 14 20:20:53.273294 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:53.273253    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"]
Jul 14 20:20:53.275312 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:20:53.275282    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b25e533_a235_48de_93e8_beca086ec6e7.slice/crio-f0889e1954e21ea74c0f566bff21cb3db12c1ea058745818786deabf2c7ccad7 WatchSource:0}: Error finding container f0889e1954e21ea74c0f566bff21cb3db12c1ea058745818786deabf2c7ccad7: Status 404 returned error can't find the container with id f0889e1954e21ea74c0f566bff21cb3db12c1ea058745818786deabf2c7ccad7
Jul 14 20:20:54.285529 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:54.285488    2581 generic.go:356] "Generic (PLEG): container finished" podID="7b25e533-a235-48de-93e8-beca086ec6e7" containerID="2f77f911f2fa3b49b09acaa07e06f6d4edc327139a29b6eca14769500e45864d" exitCode=0
Jul 14 20:20:54.285945 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:54.285570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69" event={"ID":"7b25e533-a235-48de-93e8-beca086ec6e7","Type":"ContainerDied","Data":"2f77f911f2fa3b49b09acaa07e06f6d4edc327139a29b6eca14769500e45864d"}
Jul 14 20:20:54.285945 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:54.285651    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69" event={"ID":"7b25e533-a235-48de-93e8-beca086ec6e7","Type":"ContainerStarted","Data":"f0889e1954e21ea74c0f566bff21cb3db12c1ea058745818786deabf2c7ccad7"}
Jul 14 20:20:55.439324 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:55.439290    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"
Jul 14 20:20:55.556536 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:55.556387    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b25e533-a235-48de-93e8-beca086ec6e7-maas-api-service-ca\") pod \"7b25e533-a235-48de-93e8-beca086ec6e7\" (UID: \"7b25e533-a235-48de-93e8-beca086ec6e7\") "
Jul 14 20:20:55.556787 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:55.556755    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b25e533-a235-48de-93e8-beca086ec6e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b25e533-a235-48de-93e8-beca086ec6e7" (UID: "7b25e533-a235-48de-93e8-beca086ec6e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:20:55.657633 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:55.657595    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b25e533-a235-48de-93e8-beca086ec6e7-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:20:56.300606 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:56.300577    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69"
Jul 14 20:20:56.300606 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:56.300585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6e455f-c8r69" event={"ID":"7b25e533-a235-48de-93e8-beca086ec6e7","Type":"ContainerDied","Data":"f0889e1954e21ea74c0f566bff21cb3db12c1ea058745818786deabf2c7ccad7"}
Jul 14 20:20:56.300829 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:20:56.300630    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0889e1954e21ea74c0f566bff21cb3db12c1ea058745818786deabf2c7ccad7"
Jul 14 20:21:07.166037 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:07.165996    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:21:23.801720 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:23.801685    2581 scope.go:117] "RemoveContainer" containerID="0ec94f0bd4987758305318eef870cfec23a8f2e8176a6ae0f5246f33de8f3dcc"
Jul 14 20:21:27.248395 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:27.248354    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:21:31.383805 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.383766    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"]
Jul 14 20:21:31.384433 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.384395    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b25e533-a235-48de-93e8-beca086ec6e7" containerName="revoke-keys"
Jul 14 20:21:31.384506 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.384436    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b25e533-a235-48de-93e8-beca086ec6e7" containerName="revoke-keys"
Jul 14 20:21:31.384547 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.384531    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b25e533-a235-48de-93e8-beca086ec6e7" containerName="revoke-keys"
Jul 14 20:21:31.387819 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.387797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.390977 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.390947    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ecc78d-serving-cert\""
Jul 14 20:21:31.390977 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.390965    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 20:21:31.390977 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.390985    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tfc6h\""
Jul 14 20:21:31.391842 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.391809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 20:21:31.405706 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.405669    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"]
Jul 14 20:21:31.509333 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.509299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfzdl\" (UniqueName: \"kubernetes.io/projected/e9976699-615f-4a3c-b44b-59365c6fbb53-kube-api-access-tfzdl\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.509333 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.509342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9976699-615f-4a3c-b44b-59365c6fbb53-maas-api-tls\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.509644 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.509372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9976699-615f-4a3c-b44b-59365c6fbb53-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.610966 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.610919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfzdl\" (UniqueName: \"kubernetes.io/projected/e9976699-615f-4a3c-b44b-59365c6fbb53-kube-api-access-tfzdl\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.611157 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.610977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9976699-615f-4a3c-b44b-59365c6fbb53-maas-api-tls\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.611157 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.611015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9976699-615f-4a3c-b44b-59365c6fbb53-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.611788 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.611768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9976699-615f-4a3c-b44b-59365c6fbb53-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.613701 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.613676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9976699-615f-4a3c-b44b-59365c6fbb53-maas-api-tls\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.631688 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.631649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfzdl\" (UniqueName: \"kubernetes.io/projected/e9976699-615f-4a3c-b44b-59365c6fbb53-kube-api-access-tfzdl\") pod \"maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.700621 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.700584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:31.932042 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:31.932002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"]
Jul 14 20:21:31.933287 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:21:31.933253    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9976699_615f_4a3c_b44b_59365c6fbb53.slice/crio-56535738db73321e41c371bae93e8f3c8660e8123bdd64effd41c5f49dc56bfa WatchSource:0}: Error finding container 56535738db73321e41c371bae93e8f3c8660e8123bdd64effd41c5f49dc56bfa: Status 404 returned error can't find the container with id 56535738db73321e41c371bae93e8f3c8660e8123bdd64effd41c5f49dc56bfa
Jul 14 20:21:32.539672 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:32.539632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs" event={"ID":"e9976699-615f-4a3c-b44b-59365c6fbb53","Type":"ContainerStarted","Data":"dee2c50a9cbb36e16c5a4c414fccd30ad05978e65792469e8bec19b4b5ef70b8"}
Jul 14 20:21:32.540063 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:32.539714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs" event={"ID":"e9976699-615f-4a3c-b44b-59365c6fbb53","Type":"ContainerStarted","Data":"56535738db73321e41c371bae93e8f3c8660e8123bdd64effd41c5f49dc56bfa"}
Jul 14 20:21:32.540063 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:32.539756    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:38.551949 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:38.551917    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:21:38.576101 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:38.576036    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs" podStartSLOduration=7.218102117 podStartE2EDuration="7.576013942s" totalImagesPullingTime="357.911825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:21:32.566002349 +0000 UTC m=+2590.877701806" watchObservedRunningTime="2026-07-14 20:21:38.576013942 +0000 UTC m=+2596.887713398"
Jul 14 20:21:42.475220 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:42.475181    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:21:55.055991 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.055904    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"]
Jul 14 20:21:55.060178 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.060152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.064167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.064118    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"kube-root-ca.crt\""
Jul 14 20:21:55.065076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.065052    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"auth-test-model-76db83-kserve-self-signed-certs\""
Jul 14 20:21:55.065296 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.065082    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"openshift-service-ca.crt\""
Jul 14 20:21:55.065296 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.065083    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"default-dockercfg-t6cb6\""
Jul 14 20:21:55.085022 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.084985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"]
Jul 14 20:21:55.143305 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.143256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-home\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.143305 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.143301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tmp-dir\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.143619 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.143418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tls-certs\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.143619 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.143491    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9zx\" (UniqueName: \"kubernetes.io/projected/833d291b-5012-4a27-93f7-3c4b10ca8dd5-kube-api-access-rx9zx\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.143619 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.143555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-model-cache\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.143619 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.143613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-dshm\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245006 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.244963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9zx\" (UniqueName: \"kubernetes.io/projected/833d291b-5012-4a27-93f7-3c4b10ca8dd5-kube-api-access-rx9zx\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245006 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-model-cache\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245246 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-dshm\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245246 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-home\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245246 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tmp-dir\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245246 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tls-certs\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245671 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245635    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-model-cache\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245671 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-home\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.245839 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.245680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tmp-dir\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.247439 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.247395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-dshm\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.247838 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.247817    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tls-certs\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.258705 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.258673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9zx\" (UniqueName: \"kubernetes.io/projected/833d291b-5012-4a27-93f7-3c4b10ca8dd5-kube-api-access-rx9zx\") pod \"auth-test-model-76db83-kserve-75fb88688c-glpsj\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") " pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.372592 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.372498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:21:55.539656 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.539626    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"]
Jul 14 20:21:55.540587 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:21:55.540561    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod833d291b_5012_4a27_93f7_3c4b10ca8dd5.slice/crio-d97321ce4f8d6ac7341444424322490322acd100daabfefdbd315ff358f9c29e WatchSource:0}: Error finding container d97321ce4f8d6ac7341444424322490322acd100daabfefdbd315ff358f9c29e: Status 404 returned error can't find the container with id d97321ce4f8d6ac7341444424322490322acd100daabfefdbd315ff358f9c29e
Jul 14 20:21:55.697518 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.697423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj" event={"ID":"833d291b-5012-4a27-93f7-3c4b10ca8dd5","Type":"ContainerStarted","Data":"4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e"}
Jul 14 20:21:55.697518 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.697472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj" event={"ID":"833d291b-5012-4a27-93f7-3c4b10ca8dd5","Type":"ContainerStarted","Data":"d97321ce4f8d6ac7341444424322490322acd100daabfefdbd315ff358f9c29e"}
Jul 14 20:21:55.733146 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:55.733087    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj" podStartSLOduration=0.733066432 podStartE2EDuration="733.066432ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:21:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:21:55.732667397 +0000 UTC m=+2614.044366853" watchObservedRunningTime="2026-07-14 20:21:55.733066432 +0000 UTC m=+2614.044765890"
Jul 14 20:21:58.565798 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:21:58.565749    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:22:05.372849 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:22:05.372803    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:22:05.375335 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:22:05.372893    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:22:05.386112 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:22:05.386082    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:22:05.782280 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:22:05.782247    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:22:26.261163 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:22:26.261120    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:22:39.666271 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:22:39.666229    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:22:55.639755 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:22:55.639716    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:23:13.432057 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:23:13.432014    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:23:23.509264 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:23:23.509233    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:23:23.524176 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:23:23.524150    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:23:32.026942 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:23:32.026904    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:23:51.232836 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:23:51.232795    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:24:11.233118 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:11.233063    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:24:26.212850 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:26.212815    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"]
Jul 14 20:24:29.737766 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:29.737723    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:24:41.233917 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.233842    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj" podUID="833d291b-5012-4a27-93f7-3c4b10ca8dd5" containerName="main" containerID="cri-o://4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e" gracePeriod=45
Jul 14 20:24:41.505678 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.505648    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:24:41.569070 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569035    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-model-cache\") pod \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") "
Jul 14 20:24:41.569291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569082    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-dshm\") pod \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") "
Jul 14 20:24:41.569291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569118    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tmp-dir\") pod \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") "
Jul 14 20:24:41.569291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569147    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-home\") pod \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") "
Jul 14 20:24:41.569291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569167    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx9zx\" (UniqueName: \"kubernetes.io/projected/833d291b-5012-4a27-93f7-3c4b10ca8dd5-kube-api-access-rx9zx\") pod \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") "
Jul 14 20:24:41.569291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569209    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tls-certs\") pod \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\" (UID: \"833d291b-5012-4a27-93f7-3c4b10ca8dd5\") "
Jul 14 20:24:41.569672 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-model-cache" (OuterVolumeSpecName: "model-cache") pod "833d291b-5012-4a27-93f7-3c4b10ca8dd5" (UID: "833d291b-5012-4a27-93f7-3c4b10ca8dd5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:24:41.569672 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "833d291b-5012-4a27-93f7-3c4b10ca8dd5" (UID: "833d291b-5012-4a27-93f7-3c4b10ca8dd5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:24:41.569672 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569473    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-home" (OuterVolumeSpecName: "home") pod "833d291b-5012-4a27-93f7-3c4b10ca8dd5" (UID: "833d291b-5012-4a27-93f7-3c4b10ca8dd5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:24:41.569927 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569912    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tmp-dir\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:24:41.569989 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569936    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-home\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:24:41.569989 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.569949    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-model-cache\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:24:41.572058 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.572032    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-dshm" (OuterVolumeSpecName: "dshm") pod "833d291b-5012-4a27-93f7-3c4b10ca8dd5" (UID: "833d291b-5012-4a27-93f7-3c4b10ca8dd5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:24:41.572058 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.572030    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "833d291b-5012-4a27-93f7-3c4b10ca8dd5" (UID: "833d291b-5012-4a27-93f7-3c4b10ca8dd5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:24:41.572245 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.572046    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/833d291b-5012-4a27-93f7-3c4b10ca8dd5-kube-api-access-rx9zx" (OuterVolumeSpecName: "kube-api-access-rx9zx") pod "833d291b-5012-4a27-93f7-3c4b10ca8dd5" (UID: "833d291b-5012-4a27-93f7-3c4b10ca8dd5"). InnerVolumeSpecName "kube-api-access-rx9zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:24:41.670667 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.670627    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/833d291b-5012-4a27-93f7-3c4b10ca8dd5-dshm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:24:41.670667 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.670659    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx9zx\" (UniqueName: \"kubernetes.io/projected/833d291b-5012-4a27-93f7-3c4b10ca8dd5-kube-api-access-rx9zx\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:24:41.670667 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.670672    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/833d291b-5012-4a27-93f7-3c4b10ca8dd5-tls-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:24:41.797199 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.797089    2581 generic.go:356] "Generic (PLEG): container finished" podID="833d291b-5012-4a27-93f7-3c4b10ca8dd5" containerID="4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e" exitCode=0
Jul 14 20:24:41.797199 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.797141    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"
Jul 14 20:24:41.797199 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.797184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj" event={"ID":"833d291b-5012-4a27-93f7-3c4b10ca8dd5","Type":"ContainerDied","Data":"4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e"}
Jul 14 20:24:41.797497 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.797237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj" event={"ID":"833d291b-5012-4a27-93f7-3c4b10ca8dd5","Type":"ContainerDied","Data":"d97321ce4f8d6ac7341444424322490322acd100daabfefdbd315ff358f9c29e"}
Jul 14 20:24:41.797497 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.797258    2581 scope.go:117] "RemoveContainer" containerID="4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e"
Jul 14 20:24:41.812671 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.812311    2581 scope.go:117] "RemoveContainer" containerID="4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e"
Jul 14 20:24:41.812821 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:24:41.812775    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e\": container with ID starting with 4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e not found: ID does not exist" containerID="4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e"
Jul 14 20:24:41.812889 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.812827    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e"} err="failed to get container status \"4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e\": rpc error: code = NotFound desc = could not find container \"4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e\": container with ID starting with 4ac72f161f2bdc8b77e57ce241dcfc0cfee238c3d70473bc84d73ef18d83f18e not found: ID does not exist"
Jul 14 20:24:41.826585 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.826542    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"]
Jul 14 20:24:41.830494 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:41.830461    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/auth-test-model-76db83-kserve-75fb88688c-glpsj"]
Jul 14 20:24:43.340254 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:43.340169    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="833d291b-5012-4a27-93f7-3c4b10ca8dd5" path="/var/lib/kubelet/pods/833d291b-5012-4a27-93f7-3c4b10ca8dd5/volumes"
Jul 14 20:24:45.339136 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.339103    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"]
Jul 14 20:24:45.339675 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.339558    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833d291b-5012-4a27-93f7-3c4b10ca8dd5" containerName="main"
Jul 14 20:24:45.339675 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.339573    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="833d291b-5012-4a27-93f7-3c4b10ca8dd5" containerName="main"
Jul 14 20:24:45.339675 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.339642    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="833d291b-5012-4a27-93f7-3c4b10ca8dd5" containerName="main"
Jul 14 20:24:45.344489 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.344453    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.350565 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.348086    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"openshift-service-ca.crt\""
Jul 14 20:24:45.350565 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.348089    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"kube-root-ca.crt\""
Jul 14 20:24:45.350565 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.348350    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"sub-test-model-76db83-kserve-self-signed-certs\""
Jul 14 20:24:45.350565 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.348780    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"default-dockercfg-t6cb6\""
Jul 14 20:24:45.352015 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.351990    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"]
Jul 14 20:24:45.409097 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.409058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n58q7\" (UniqueName: \"kubernetes.io/projected/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-kube-api-access-n58q7\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.409097 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.409097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-dshm\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.409378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.409125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-model-cache\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.409378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.409197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-home\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.409378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.409282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tmp-dir\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.409378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.409314    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tls-certs\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510391 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tmp-dir\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510391 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tls-certs\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510702 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n58q7\" (UniqueName: \"kubernetes.io/projected/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-kube-api-access-n58q7\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510702 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-dshm\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510702 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-model-cache\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510702 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-home\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510938 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tmp-dir\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.510938 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.510919    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-home\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.511484 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.511452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-model-cache\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.513943 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.513903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-dshm\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.514217 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.514196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tls-certs\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.519333 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.519307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n58q7\" (UniqueName: \"kubernetes.io/projected/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-kube-api-access-n58q7\") pod \"sub-test-model-76db83-kserve-755697d88c-prswv\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") " pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.658373 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.658274    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:45.816567 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.816537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"]
Jul 14 20:24:45.818372 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:24:45.818337    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdf4a0b7_e2da_4ed8_9fd3_2da185d2be1c.slice/crio-874fb289420e1c400c5721259ddcfa53141d35a7a2ba0b413bb33f533d677608 WatchSource:0}: Error finding container 874fb289420e1c400c5721259ddcfa53141d35a7a2ba0b413bb33f533d677608: Status 404 returned error can't find the container with id 874fb289420e1c400c5721259ddcfa53141d35a7a2ba0b413bb33f533d677608
Jul 14 20:24:45.835080 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:45.835042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv" event={"ID":"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c","Type":"ContainerStarted","Data":"874fb289420e1c400c5721259ddcfa53141d35a7a2ba0b413bb33f533d677608"}
Jul 14 20:24:46.730044 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:46.730001    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:24:46.843263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:46.843224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv" event={"ID":"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c","Type":"ContainerStarted","Data":"53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333"}
Jul 14 20:24:46.859262 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:46.859200    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv" podStartSLOduration=1.8591797780000001 podStartE2EDuration="1.859179778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:24:46.858796683 +0000 UTC m=+2785.170496139" watchObservedRunningTime="2026-07-14 20:24:46.859179778 +0000 UTC m=+2785.170879235"
Jul 14 20:24:55.658973 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:55.658933    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:55.658973 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:55.658984    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:55.672836 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:55.672801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:55.916258 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:55.916169    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:24:57.131466 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:24:57.131422    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:25:21.737270 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:25:21.737225    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:25:36.933193 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:25:36.933151    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:25:57.339030 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:25:57.338992    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:26:02.426884 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:02.426847    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"]
Jul 14 20:26:15.513761 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.513674    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"]
Jul 14 20:26:15.517853 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.517827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.520276 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.520255    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-76db83\"/\"rate-test-model-76db83-kserve-self-signed-certs\""
Jul 14 20:26:15.528446 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.528420    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"]
Jul 14 20:26:15.617203 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.617163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-dshm\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.617448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.617218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a2375b-2956-4974-adaa-937c5649c3b1-tls-certs\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.617448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.617251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-tmp-dir\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.617448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.617293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-home\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.617448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.617320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-model-cache\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.617448 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.617422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvd9m\" (UniqueName: \"kubernetes.io/projected/d0a2375b-2956-4974-adaa-937c5649c3b1-kube-api-access-mvd9m\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718277 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvd9m\" (UniqueName: \"kubernetes.io/projected/d0a2375b-2956-4974-adaa-937c5649c3b1-kube-api-access-mvd9m\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718540 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-dshm\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718540 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a2375b-2956-4974-adaa-937c5649c3b1-tls-certs\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718540 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-tmp-dir\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718540 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-home\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718540 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-model-cache\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718844 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-tmp-dir\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718913 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-home\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.718966 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.718914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-model-cache\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.720848 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.720819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-dshm\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.720972 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.720959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a2375b-2956-4974-adaa-937c5649c3b1-tls-certs\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.726914 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.726886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvd9m\" (UniqueName: \"kubernetes.io/projected/d0a2375b-2956-4974-adaa-937c5649c3b1-kube-api-access-mvd9m\") pod \"rate-test-model-76db83-kserve-8f44fcd99-fzl4b\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") " pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.828809 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.828708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:15.987204 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:15.987161    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"]
Jul 14 20:26:15.987829 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:26:15.987801    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0a2375b_2956_4974_adaa_937c5649c3b1.slice/crio-e37ef4fa0642accd6e56943db96224fc7c0933a2a2dde2c2c2dc5b31b37c79f6 WatchSource:0}: Error finding container e37ef4fa0642accd6e56943db96224fc7c0933a2a2dde2c2c2dc5b31b37c79f6: Status 404 returned error can't find the container with id e37ef4fa0642accd6e56943db96224fc7c0933a2a2dde2c2c2dc5b31b37c79f6
Jul 14 20:26:16.337291 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:16.337254    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:26:16.437946 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:16.437897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b" event={"ID":"d0a2375b-2956-4974-adaa-937c5649c3b1","Type":"ContainerStarted","Data":"45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8"}
Jul 14 20:26:16.438149 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:16.437959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b" event={"ID":"d0a2375b-2956-4974-adaa-937c5649c3b1","Type":"ContainerStarted","Data":"e37ef4fa0642accd6e56943db96224fc7c0933a2a2dde2c2c2dc5b31b37c79f6"}
Jul 14 20:26:16.457167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:16.457106    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b" podStartSLOduration=1.4570869069999999 podStartE2EDuration="1.457086907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:16.456045057 +0000 UTC m=+2874.767744510" watchObservedRunningTime="2026-07-14 20:26:16.457086907 +0000 UTC m=+2874.768786362"
Jul 14 20:26:17.447461 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.447413    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv" podUID="fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" containerName="main" containerID="cri-o://53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333" gracePeriod=45
Jul 14 20:26:17.727201 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.727168    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:26:17.840575 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-dshm\") pod \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") "
Jul 14 20:26:17.840575 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840583    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-home\") pod \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") "
Jul 14 20:26:17.840857 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840616    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-model-cache\") pod \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") "
Jul 14 20:26:17.840857 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840645    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n58q7\" (UniqueName: \"kubernetes.io/projected/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-kube-api-access-n58q7\") pod \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") "
Jul 14 20:26:17.840857 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840701    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tmp-dir\") pod \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") "
Jul 14 20:26:17.840857 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840719    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tls-certs\") pod \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\" (UID: \"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c\") "
Jul 14 20:26:17.841058 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840902    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-home" (OuterVolumeSpecName: "home") pod "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" (UID: "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:26:17.841058 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840916    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-model-cache" (OuterVolumeSpecName: "model-cache") pod "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" (UID: "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:26:17.841058 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.840956    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" (UID: "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:26:17.841209 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.841107    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-home\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:26:17.841209 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.841127    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-model-cache\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:26:17.841209 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.841143    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tmp-dir\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:26:17.842949 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.842921    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-dshm" (OuterVolumeSpecName: "dshm") pod "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" (UID: "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:26:17.842949 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.842944    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-kube-api-access-n58q7" (OuterVolumeSpecName: "kube-api-access-n58q7") pod "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" (UID: "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c"). InnerVolumeSpecName "kube-api-access-n58q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:26:17.843123 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.843001    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" (UID: "fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:26:17.942100 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.942043    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-dshm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:26:17.942100 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.942092    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n58q7\" (UniqueName: \"kubernetes.io/projected/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-kube-api-access-n58q7\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:26:17.942100 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:17.942104    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c-tls-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:26:18.455372 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.455333    2581 generic.go:356] "Generic (PLEG): container finished" podID="fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" containerID="53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333" exitCode=0
Jul 14 20:26:18.455891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.455381    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"
Jul 14 20:26:18.455891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.455434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv" event={"ID":"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c","Type":"ContainerDied","Data":"53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333"}
Jul 14 20:26:18.455891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.455479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv" event={"ID":"fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c","Type":"ContainerDied","Data":"874fb289420e1c400c5721259ddcfa53141d35a7a2ba0b413bb33f533d677608"}
Jul 14 20:26:18.455891 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.455504    2581 scope.go:117] "RemoveContainer" containerID="53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333"
Jul 14 20:26:18.466182 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.466157    2581 scope.go:117] "RemoveContainer" containerID="53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333"
Jul 14 20:26:18.466575 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:26:18.466552    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333\": container with ID starting with 53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333 not found: ID does not exist" containerID="53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333"
Jul 14 20:26:18.466689 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.466587    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333"} err="failed to get container status \"53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333\": rpc error: code = NotFound desc = could not find container \"53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333\": container with ID starting with 53a60db9ce3dc54704ed7d0788b44a9e93bbba86a44c81a0aa1f8c6ff9f85333 not found: ID does not exist"
Jul 14 20:26:18.481614 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.481539    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"]
Jul 14 20:26:18.485250 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:18.485220    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/sub-test-model-76db83-kserve-755697d88c-prswv"]
Jul 14 20:26:19.340495 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:19.340458    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" path="/var/lib/kubelet/pods/fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c/volumes"
Jul 14 20:26:25.828992 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:25.828951    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:25.829533 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:25.829047    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:25.842117 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:25.842091    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:26.522137 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:26.522107    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:26:29.932496 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:29.932446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:26:38.739856 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.739817    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"]
Jul 14 20:26:38.740421 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.740387    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" containerName="main"
Jul 14 20:26:38.740472 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.740423    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" containerName="main"
Jul 14 20:26:38.740528 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.740519    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdf4a0b7-e2da-4ed8-9fd3-2da185d2be1c" containerName="main"
Jul 14 20:26:38.743905 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.743883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.747701 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.747673    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"openshift-service-ca.crt\""
Jul 14 20:26:38.747992 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.747975    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"rate-test-model-ecc78d-kserve-self-signed-certs\""
Jul 14 20:26:38.748505 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.748486    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"default-dockercfg-gz8ps\""
Jul 14 20:26:38.748611 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.748534    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ecc78d\"/\"kube-root-ca.crt\""
Jul 14 20:26:38.758120 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.758087    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"]
Jul 14 20:26:38.855289 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.855247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-tmp-dir\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.855289 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.855289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-dshm\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.855558 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.855371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca0f0036-19a3-4250-918b-fb9d4033d31c-tls-certs\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.855558 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.855479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwp8w\" (UniqueName: \"kubernetes.io/projected/ca0f0036-19a3-4250-918b-fb9d4033d31c-kube-api-access-dwp8w\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.855558 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.855543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-home\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.855664 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.855579    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-model-cache\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.956928 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.956884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca0f0036-19a3-4250-918b-fb9d4033d31c-tls-certs\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957089 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.956958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwp8w\" (UniqueName: \"kubernetes.io/projected/ca0f0036-19a3-4250-918b-fb9d4033d31c-kube-api-access-dwp8w\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957089 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.957025    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-home\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957089 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.957064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-model-cache\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957231 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.957102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-tmp-dir\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957231 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.957134    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-dshm\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957576 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.957548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-home\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957576 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.957569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-model-cache\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.957742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.957588    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-tmp-dir\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.959529 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.959498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-dshm\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.959683 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.959666    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca0f0036-19a3-4250-918b-fb9d4033d31c-tls-certs\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:38.968307 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:38.968273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwp8w\" (UniqueName: \"kubernetes.io/projected/ca0f0036-19a3-4250-918b-fb9d4033d31c-kube-api-access-dwp8w\") pod \"rate-test-model-ecc78d-kserve-79f7744b99-f69dg\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") " pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:39.054235 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:39.054123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:39.214162 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:39.214070    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"]
Jul 14 20:26:39.216902 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:26:39.216851    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca0f0036_19a3_4250_918b_fb9d4033d31c.slice/crio-d2852fece8aff7ee6359ba484bb15b4e95ebaca179c86f937569d9a1d794ca71 WatchSource:0}: Error finding container d2852fece8aff7ee6359ba484bb15b4e95ebaca179c86f937569d9a1d794ca71: Status 404 returned error can't find the container with id d2852fece8aff7ee6359ba484bb15b4e95ebaca179c86f937569d9a1d794ca71
Jul 14 20:26:39.600599 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:39.600558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg" event={"ID":"ca0f0036-19a3-4250-918b-fb9d4033d31c","Type":"ContainerStarted","Data":"7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30"}
Jul 14 20:26:39.600804 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:39.600609    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg" event={"ID":"ca0f0036-19a3-4250-918b-fb9d4033d31c","Type":"ContainerStarted","Data":"d2852fece8aff7ee6359ba484bb15b4e95ebaca179c86f937569d9a1d794ca71"}
Jul 14 20:26:39.620988 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:39.620933    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg" podStartSLOduration=1.6209175660000001 podStartE2EDuration="1.620917566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:39.620057609 +0000 UTC m=+2897.931757069" watchObservedRunningTime="2026-07-14 20:26:39.620917566 +0000 UTC m=+2897.932617021"
Jul 14 20:26:46.442673 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:46.442633    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:26:49.055262 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:49.055215    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:49.055774 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:49.055275    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:49.069060 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:49.069031    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:26:49.675773 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:26:49.675742    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:27:07.729656 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:07.729613    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:27:23.733005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:23.732969    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:27:30.272206 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:30.272161    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"]
Jul 14 20:27:31.248467 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:31.248421    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"]
Jul 14 20:27:32.254306 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.254264    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-139758697819696"]
Jul 14 20:27:32.258087 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.258062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697819696"
Jul 14 20:27:32.260768 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.260745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hjhkn\""
Jul 14 20:27:32.264131 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.264096    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697819696"]
Jul 14 20:27:32.397297 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.397261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxjx5\" (UniqueName: \"kubernetes.io/projected/f169ab76-d676-49a4-bdc8-4cda9463a0ce-kube-api-access-vxjx5\") pod \"test-curl-7491-139758697819696\" (UID: \"f169ab76-d676-49a4-bdc8-4cda9463a0ce\") " pod="opendatahub/test-curl-7491-139758697819696"
Jul 14 20:27:32.498350 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.498302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxjx5\" (UniqueName: \"kubernetes.io/projected/f169ab76-d676-49a4-bdc8-4cda9463a0ce-kube-api-access-vxjx5\") pod \"test-curl-7491-139758697819696\" (UID: \"f169ab76-d676-49a4-bdc8-4cda9463a0ce\") " pod="opendatahub/test-curl-7491-139758697819696"
Jul 14 20:27:32.506811 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.506731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxjx5\" (UniqueName: \"kubernetes.io/projected/f169ab76-d676-49a4-bdc8-4cda9463a0ce-kube-api-access-vxjx5\") pod \"test-curl-7491-139758697819696\" (UID: \"f169ab76-d676-49a4-bdc8-4cda9463a0ce\") " pod="opendatahub/test-curl-7491-139758697819696"
Jul 14 20:27:32.573953 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.573896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697819696"
Jul 14 20:27:32.721894 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.721861    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697819696"]
Jul 14 20:27:32.723687 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:27:32.723654    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf169ab76_d676_49a4_bdc8_4cda9463a0ce.slice/crio-26f89b00a5e2fe02b096d8089ceda92cd1e45c32b89b43cb464126bf0444e60a WatchSource:0}: Error finding container 26f89b00a5e2fe02b096d8089ceda92cd1e45c32b89b43cb464126bf0444e60a: Status 404 returned error can't find the container with id 26f89b00a5e2fe02b096d8089ceda92cd1e45c32b89b43cb464126bf0444e60a
Jul 14 20:27:32.725371 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.725350    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 20:27:32.951319 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:32.951280    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697819696" event={"ID":"f169ab76-d676-49a4-bdc8-4cda9463a0ce","Type":"ContainerStarted","Data":"26f89b00a5e2fe02b096d8089ceda92cd1e45c32b89b43cb464126bf0444e60a"}
Jul 14 20:27:33.959760 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:33.959718    2581 generic.go:356] "Generic (PLEG): container finished" podID="f169ab76-d676-49a4-bdc8-4cda9463a0ce" containerID="460beaba9e5a9edd8662dd669255ceef4195955719a0bcb3abf8e1bdc4311117" exitCode=0
Jul 14 20:27:33.960190 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:33.959801    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697819696" event={"ID":"f169ab76-d676-49a4-bdc8-4cda9463a0ce","Type":"ContainerDied","Data":"460beaba9e5a9edd8662dd669255ceef4195955719a0bcb3abf8e1bdc4311117"}
Jul 14 20:27:35.119431 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.119377    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697819696"
Jul 14 20:27:35.156192 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.156156    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-139758697819696_f169ab76-d676-49a4-bdc8-4cda9463a0ce/test-curl-7491-139758697819696/0.log"
Jul 14 20:27:35.183176 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.183140    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-139758697819696"]
Jul 14 20:27:35.189792 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.189756    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-139758697819696"]
Jul 14 20:27:35.230196 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.230099    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxjx5\" (UniqueName: \"kubernetes.io/projected/f169ab76-d676-49a4-bdc8-4cda9463a0ce-kube-api-access-vxjx5\") pod \"f169ab76-d676-49a4-bdc8-4cda9463a0ce\" (UID: \"f169ab76-d676-49a4-bdc8-4cda9463a0ce\") "
Jul 14 20:27:35.232395 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.232365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f169ab76-d676-49a4-bdc8-4cda9463a0ce-kube-api-access-vxjx5" (OuterVolumeSpecName: "kube-api-access-vxjx5") pod "f169ab76-d676-49a4-bdc8-4cda9463a0ce" (UID: "f169ab76-d676-49a4-bdc8-4cda9463a0ce"). InnerVolumeSpecName "kube-api-access-vxjx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:35.316896 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.316852    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-139758697817008"]
Jul 14 20:27:35.317375 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.317360    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f169ab76-d676-49a4-bdc8-4cda9463a0ce" containerName="test-curl-7491-139758697819696"
Jul 14 20:27:35.317469 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.317377    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f169ab76-d676-49a4-bdc8-4cda9463a0ce" containerName="test-curl-7491-139758697819696"
Jul 14 20:27:35.317525 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.317498    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f169ab76-d676-49a4-bdc8-4cda9463a0ce" containerName="test-curl-7491-139758697819696"
Jul 14 20:27:35.320934 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.320905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:35.324637 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.324606    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697817008"]
Jul 14 20:27:35.331634 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.331604    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxjx5\" (UniqueName: \"kubernetes.io/projected/f169ab76-d676-49a4-bdc8-4cda9463a0ce-kube-api-access-vxjx5\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:35.350832 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.344209    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f169ab76-d676-49a4-bdc8-4cda9463a0ce" path="/var/lib/kubelet/pods/f169ab76-d676-49a4-bdc8-4cda9463a0ce/volumes"
Jul 14 20:27:35.432612 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.432570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r94xd\" (UniqueName: \"kubernetes.io/projected/33e23d70-95b1-44cc-a426-88ecbfdf7e25-kube-api-access-r94xd\") pod \"test-curl-7491-139758697817008\" (UID: \"33e23d70-95b1-44cc-a426-88ecbfdf7e25\") " pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:35.534274 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.534162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r94xd\" (UniqueName: \"kubernetes.io/projected/33e23d70-95b1-44cc-a426-88ecbfdf7e25-kube-api-access-r94xd\") pod \"test-curl-7491-139758697817008\" (UID: \"33e23d70-95b1-44cc-a426-88ecbfdf7e25\") " pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:35.543521 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.543484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r94xd\" (UniqueName: \"kubernetes.io/projected/33e23d70-95b1-44cc-a426-88ecbfdf7e25-kube-api-access-r94xd\") pod \"test-curl-7491-139758697817008\" (UID: \"33e23d70-95b1-44cc-a426-88ecbfdf7e25\") " pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:35.632716 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.632659    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:35.781839 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.781790    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697817008"]
Jul 14 20:27:35.782535 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:27:35.782505    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33e23d70_95b1_44cc_a426_88ecbfdf7e25.slice/crio-24c30e949751ee737c77f79c38ebbe3ebfd9315fd4c40adde105e19a25d2ec59 WatchSource:0}: Error finding container 24c30e949751ee737c77f79c38ebbe3ebfd9315fd4c40adde105e19a25d2ec59: Status 404 returned error can't find the container with id 24c30e949751ee737c77f79c38ebbe3ebfd9315fd4c40adde105e19a25d2ec59
Jul 14 20:27:35.975785 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.975716    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697817008" event={"ID":"33e23d70-95b1-44cc-a426-88ecbfdf7e25","Type":"ContainerStarted","Data":"24c30e949751ee737c77f79c38ebbe3ebfd9315fd4c40adde105e19a25d2ec59"}
Jul 14 20:27:35.977361 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.977331    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697819696"
Jul 14 20:27:35.977529 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:35.977377    2581 scope.go:117] "RemoveContainer" containerID="460beaba9e5a9edd8662dd669255ceef4195955719a0bcb3abf8e1bdc4311117"
Jul 14 20:27:36.986357 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:36.986316    2581 generic.go:356] "Generic (PLEG): container finished" podID="33e23d70-95b1-44cc-a426-88ecbfdf7e25" containerID="aac44c90434b168309b6375e8332adf65258a9d97ba4c7094cc9b5e75b143aca" exitCode=0
Jul 14 20:27:36.986856 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:36.986412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697817008" event={"ID":"33e23d70-95b1-44cc-a426-88ecbfdf7e25","Type":"ContainerDied","Data":"aac44c90434b168309b6375e8332adf65258a9d97ba4c7094cc9b5e75b143aca"}
Jul 14 20:27:38.142818 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.142790    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:38.163584 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.163540    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-139758697817008_33e23d70-95b1-44cc-a426-88ecbfdf7e25/test-curl-7491-139758697817008/0.log"
Jul 14 20:27:38.187849 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.187812    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-139758697817008"]
Jul 14 20:27:38.191898 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.191856    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-139758697817008"]
Jul 14 20:27:38.262767 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.262662    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r94xd\" (UniqueName: \"kubernetes.io/projected/33e23d70-95b1-44cc-a426-88ecbfdf7e25-kube-api-access-r94xd\") pod \"33e23d70-95b1-44cc-a426-88ecbfdf7e25\" (UID: \"33e23d70-95b1-44cc-a426-88ecbfdf7e25\") "
Jul 14 20:27:38.264989 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.264958    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e23d70-95b1-44cc-a426-88ecbfdf7e25-kube-api-access-r94xd" (OuterVolumeSpecName: "kube-api-access-r94xd") pod "33e23d70-95b1-44cc-a426-88ecbfdf7e25" (UID: "33e23d70-95b1-44cc-a426-88ecbfdf7e25"). InnerVolumeSpecName "kube-api-access-r94xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:38.359988 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.359947    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:38.360427 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.360411    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e23d70-95b1-44cc-a426-88ecbfdf7e25" containerName="test-curl-7491-139758697817008"
Jul 14 20:27:38.360427 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.360427    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e23d70-95b1-44cc-a426-88ecbfdf7e25" containerName="test-curl-7491-139758697817008"
Jul 14 20:27:38.360515 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.360504    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e23d70-95b1-44cc-a426-88ecbfdf7e25" containerName="test-curl-7491-139758697817008"
Jul 14 20:27:38.363720 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.363703    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:38.364272 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.364251    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r94xd\" (UniqueName: \"kubernetes.io/projected/33e23d70-95b1-44cc-a426-88ecbfdf7e25-kube-api-access-r94xd\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:38.374298 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.374266    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:38.465585 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.465536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nq4j\" (UniqueName: \"kubernetes.io/projected/6e11094d-8cc4-4d7d-b89a-b41259a60d1e-kube-api-access-5nq4j\") pod \"test-curl-7491-139758697781840\" (UID: \"6e11094d-8cc4-4d7d-b89a-b41259a60d1e\") " pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:38.567241 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.567136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nq4j\" (UniqueName: \"kubernetes.io/projected/6e11094d-8cc4-4d7d-b89a-b41259a60d1e-kube-api-access-5nq4j\") pod \"test-curl-7491-139758697781840\" (UID: \"6e11094d-8cc4-4d7d-b89a-b41259a60d1e\") " pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:38.576238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.576208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nq4j\" (UniqueName: \"kubernetes.io/projected/6e11094d-8cc4-4d7d-b89a-b41259a60d1e-kube-api-access-5nq4j\") pod \"test-curl-7491-139758697781840\" (UID: \"6e11094d-8cc4-4d7d-b89a-b41259a60d1e\") " pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:38.674722 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.674680    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:38.825000 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:38.824969    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:38.826472 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:27:38.826442    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e11094d_8cc4_4d7d_b89a_b41259a60d1e.slice/crio-433fded877a3f5e10daaab5654232b07f43d04a0d7b8c75c6f4633055597bd99 WatchSource:0}: Error finding container 433fded877a3f5e10daaab5654232b07f43d04a0d7b8c75c6f4633055597bd99: Status 404 returned error can't find the container with id 433fded877a3f5e10daaab5654232b07f43d04a0d7b8c75c6f4633055597bd99
Jul 14 20:27:39.002803 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:39.002763    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697781840" event={"ID":"6e11094d-8cc4-4d7d-b89a-b41259a60d1e","Type":"ContainerStarted","Data":"433fded877a3f5e10daaab5654232b07f43d04a0d7b8c75c6f4633055597bd99"}
Jul 14 20:27:39.004416 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:39.004376    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:39.004579 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:39.004437    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24c30e949751ee737c77f79c38ebbe3ebfd9315fd4c40adde105e19a25d2ec59"
Jul 14 20:27:39.015706 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:27:39.015669    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-139758697817008\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="33e23d70-95b1-44cc-a426-88ecbfdf7e25" pod="opendatahub/test-curl-7491-139758697817008"
Jul 14 20:27:39.338598 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:39.338560    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e23d70-95b1-44cc-a426-88ecbfdf7e25" path="/var/lib/kubelet/pods/33e23d70-95b1-44cc-a426-88ecbfdf7e25/volumes"
Jul 14 20:27:39.434337 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:39.434243    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:27:40.012791 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:40.012759    2581 generic.go:356] "Generic (PLEG): container finished" podID="6e11094d-8cc4-4d7d-b89a-b41259a60d1e" containerID="1d90e3e4a319f8d2e8e26625968a664beb543b9fdb4e6a741416f870991aaacf" exitCode=0
Jul 14 20:27:40.013027 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:40.012846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697781840" event={"ID":"6e11094d-8cc4-4d7d-b89a-b41259a60d1e","Type":"ContainerDied","Data":"1d90e3e4a319f8d2e8e26625968a664beb543b9fdb4e6a741416f870991aaacf"}
Jul 14 20:27:41.175661 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.175628    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:41.196897 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.196864    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-139758697781840_6e11094d-8cc4-4d7d-b89a-b41259a60d1e/test-curl-7491-139758697781840/0.log"
Jul 14 20:27:41.225414 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.225339    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:41.230392 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.230359    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:41.293764 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.293670    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nq4j\" (UniqueName: \"kubernetes.io/projected/6e11094d-8cc4-4d7d-b89a-b41259a60d1e-kube-api-access-5nq4j\") pod \"6e11094d-8cc4-4d7d-b89a-b41259a60d1e\" (UID: \"6e11094d-8cc4-4d7d-b89a-b41259a60d1e\") "
Jul 14 20:27:41.295831 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.295797    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e11094d-8cc4-4d7d-b89a-b41259a60d1e-kube-api-access-5nq4j" (OuterVolumeSpecName: "kube-api-access-5nq4j") pod "6e11094d-8cc4-4d7d-b89a-b41259a60d1e" (UID: "6e11094d-8cc4-4d7d-b89a-b41259a60d1e"). InnerVolumeSpecName "kube-api-access-5nq4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:41.342992 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.342953    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e11094d-8cc4-4d7d-b89a-b41259a60d1e" path="/var/lib/kubelet/pods/6e11094d-8cc4-4d7d-b89a-b41259a60d1e/volumes"
Jul 14 20:27:41.352081 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.352058    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:41.352512 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.352499    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e11094d-8cc4-4d7d-b89a-b41259a60d1e" containerName="test-curl-7491-139758697781840"
Jul 14 20:27:41.352567 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.352515    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e11094d-8cc4-4d7d-b89a-b41259a60d1e" containerName="test-curl-7491-139758697781840"
Jul 14 20:27:41.352620 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.352611    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e11094d-8cc4-4d7d-b89a-b41259a60d1e" containerName="test-curl-7491-139758697781840"
Jul 14 20:27:41.355803 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.355786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:41.362351 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.362320    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:41.394538 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.394495    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nq4j\" (UniqueName: \"kubernetes.io/projected/6e11094d-8cc4-4d7d-b89a-b41259a60d1e-kube-api-access-5nq4j\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:41.495765 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.495722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hv8m\" (UniqueName: \"kubernetes.io/projected/d3df8ddc-b2a6-43a1-80f1-d42d629d834b-kube-api-access-7hv8m\") pod \"test-curl-7491-139758697781840\" (UID: \"d3df8ddc-b2a6-43a1-80f1-d42d629d834b\") " pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:41.596854 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.596752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hv8m\" (UniqueName: \"kubernetes.io/projected/d3df8ddc-b2a6-43a1-80f1-d42d629d834b-kube-api-access-7hv8m\") pod \"test-curl-7491-139758697781840\" (UID: \"d3df8ddc-b2a6-43a1-80f1-d42d629d834b\") " pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:41.606701 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.605926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hv8m\" (UniqueName: \"kubernetes.io/projected/d3df8ddc-b2a6-43a1-80f1-d42d629d834b-kube-api-access-7hv8m\") pod \"test-curl-7491-139758697781840\" (UID: \"d3df8ddc-b2a6-43a1-80f1-d42d629d834b\") " pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:41.667497 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.667463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:41.815375 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:41.815345    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:41.816843 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:27:41.816816    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3df8ddc_b2a6_43a1_80f1_d42d629d834b.slice/crio-7a92ad21de67bece08ddfbf07980a557e0033bb22a64eb4db0122d4460e00033 WatchSource:0}: Error finding container 7a92ad21de67bece08ddfbf07980a557e0033bb22a64eb4db0122d4460e00033: Status 404 returned error can't find the container with id 7a92ad21de67bece08ddfbf07980a557e0033bb22a64eb4db0122d4460e00033
Jul 14 20:27:42.035742 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:42.035707    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697781840" event={"ID":"d3df8ddc-b2a6-43a1-80f1-d42d629d834b","Type":"ContainerStarted","Data":"7a92ad21de67bece08ddfbf07980a557e0033bb22a64eb4db0122d4460e00033"}
Jul 14 20:27:42.037076 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:42.037052    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:42.037215 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:42.037101    2581 scope.go:117] "RemoveContainer" containerID="1d90e3e4a319f8d2e8e26625968a664beb543b9fdb4e6a741416f870991aaacf"
Jul 14 20:27:43.047115 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:43.047073    2581 generic.go:356] "Generic (PLEG): container finished" podID="d3df8ddc-b2a6-43a1-80f1-d42d629d834b" containerID="720c07d49eccc5579de06a937182c25f1503a743a6bda5598e622898d6009f09" exitCode=0
Jul 14 20:27:43.047599 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:43.047158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697781840" event={"ID":"d3df8ddc-b2a6-43a1-80f1-d42d629d834b","Type":"ContainerDied","Data":"720c07d49eccc5579de06a937182c25f1503a743a6bda5598e622898d6009f09"}
Jul 14 20:27:44.212934 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.212908    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:44.235602 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.235568    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-139758697781840_d3df8ddc-b2a6-43a1-80f1-d42d629d834b/test-curl-7491-139758697781840/0.log"
Jul 14 20:27:44.262981 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.262924    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:44.268008 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.267976    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-139758697781840"]
Jul 14 20:27:44.326182 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.326078    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hv8m\" (UniqueName: \"kubernetes.io/projected/d3df8ddc-b2a6-43a1-80f1-d42d629d834b-kube-api-access-7hv8m\") pod \"d3df8ddc-b2a6-43a1-80f1-d42d629d834b\" (UID: \"d3df8ddc-b2a6-43a1-80f1-d42d629d834b\") "
Jul 14 20:27:44.328220 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.328183    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3df8ddc-b2a6-43a1-80f1-d42d629d834b-kube-api-access-7hv8m" (OuterVolumeSpecName: "kube-api-access-7hv8m") pod "d3df8ddc-b2a6-43a1-80f1-d42d629d834b" (UID: "d3df8ddc-b2a6-43a1-80f1-d42d629d834b"). InnerVolumeSpecName "kube-api-access-7hv8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:44.391727 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.391690    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:44.392151 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.392137    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3df8ddc-b2a6-43a1-80f1-d42d629d834b" containerName="test-curl-7491-139758697781840"
Jul 14 20:27:44.392151 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.392152    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3df8ddc-b2a6-43a1-80f1-d42d629d834b" containerName="test-curl-7491-139758697781840"
Jul 14 20:27:44.392238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.392226    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3df8ddc-b2a6-43a1-80f1-d42d629d834b" containerName="test-curl-7491-139758697781840"
Jul 14 20:27:44.395377 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.395360    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:44.400561 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.400535    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:44.427173 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.427139    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hv8m\" (UniqueName: \"kubernetes.io/projected/d3df8ddc-b2a6-43a1-80f1-d42d629d834b-kube-api-access-7hv8m\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:44.528587 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.528547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sncnc\" (UniqueName: \"kubernetes.io/projected/a87c54dd-4607-43fa-8850-b16ea68502cf-kube-api-access-sncnc\") pod \"test-curl-7491-139758697782848\" (UID: \"a87c54dd-4607-43fa-8850-b16ea68502cf\") " pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:44.629942 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.629826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sncnc\" (UniqueName: \"kubernetes.io/projected/a87c54dd-4607-43fa-8850-b16ea68502cf-kube-api-access-sncnc\") pod \"test-curl-7491-139758697782848\" (UID: \"a87c54dd-4607-43fa-8850-b16ea68502cf\") " pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:44.638359 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.638333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sncnc\" (UniqueName: \"kubernetes.io/projected/a87c54dd-4607-43fa-8850-b16ea68502cf-kube-api-access-sncnc\") pod \"test-curl-7491-139758697782848\" (UID: \"a87c54dd-4607-43fa-8850-b16ea68502cf\") " pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:44.705588 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.705553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:44.853214 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:44.853185    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:44.854649 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:27:44.854615    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda87c54dd_4607_43fa_8850_b16ea68502cf.slice/crio-117a4b40c019090e0c56950f09fdc280df36d9f3cef9bf7b8b41ab67a27f8836 WatchSource:0}: Error finding container 117a4b40c019090e0c56950f09fdc280df36d9f3cef9bf7b8b41ab67a27f8836: Status 404 returned error can't find the container with id 117a4b40c019090e0c56950f09fdc280df36d9f3cef9bf7b8b41ab67a27f8836
Jul 14 20:27:45.061599 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.061554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697782848" event={"ID":"a87c54dd-4607-43fa-8850-b16ea68502cf","Type":"ContainerStarted","Data":"117a4b40c019090e0c56950f09fdc280df36d9f3cef9bf7b8b41ab67a27f8836"}
Jul 14 20:27:45.062973 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.062946    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:45.062973 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.062960    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a92ad21de67bece08ddfbf07980a557e0033bb22a64eb4db0122d4460e00033"
Jul 14 20:27:45.074229 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:27:45.074196    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-139758697781840\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="d3df8ddc-b2a6-43a1-80f1-d42d629d834b" pod="opendatahub/test-curl-7491-139758697781840"
Jul 14 20:27:45.293044 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.293004    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b" podUID="d0a2375b-2956-4974-adaa-937c5649c3b1" containerName="main" containerID="cri-o://45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8" gracePeriod=45
Jul 14 20:27:45.339644 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.339615    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3df8ddc-b2a6-43a1-80f1-d42d629d834b" path="/var/lib/kubelet/pods/d3df8ddc-b2a6-43a1-80f1-d42d629d834b/volumes"
Jul 14 20:27:45.566443 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.566383    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:27:45.740630 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.740590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-dshm\") pod \"d0a2375b-2956-4974-adaa-937c5649c3b1\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") "
Jul 14 20:27:45.740630 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.740635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-home\") pod \"d0a2375b-2956-4974-adaa-937c5649c3b1\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") "
Jul 14 20:27:45.740883 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.740680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-tmp-dir\") pod \"d0a2375b-2956-4974-adaa-937c5649c3b1\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") "
Jul 14 20:27:45.740883 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.740761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a2375b-2956-4974-adaa-937c5649c3b1-tls-certs\") pod \"d0a2375b-2956-4974-adaa-937c5649c3b1\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") "
Jul 14 20:27:45.740883 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.740792    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-model-cache\") pod \"d0a2375b-2956-4974-adaa-937c5649c3b1\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") "
Jul 14 20:27:45.740883 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.740829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvd9m\" (UniqueName: \"kubernetes.io/projected/d0a2375b-2956-4974-adaa-937c5649c3b1-kube-api-access-mvd9m\") pod \"d0a2375b-2956-4974-adaa-937c5649c3b1\" (UID: \"d0a2375b-2956-4974-adaa-937c5649c3b1\") "
Jul 14 20:27:45.741096 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.740975    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-home" (OuterVolumeSpecName: "home") pod "d0a2375b-2956-4974-adaa-937c5649c3b1" (UID: "d0a2375b-2956-4974-adaa-937c5649c3b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:45.741096 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.741007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0a2375b-2956-4974-adaa-937c5649c3b1" (UID: "d0a2375b-2956-4974-adaa-937c5649c3b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:45.741210 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.741107    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0a2375b-2956-4974-adaa-937c5649c3b1" (UID: "d0a2375b-2956-4974-adaa-937c5649c3b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:45.741261 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.741244    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-tmp-dir\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:45.741310 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.741264    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-model-cache\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:45.741310 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.741279    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-home\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:45.742908 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.742884    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-dshm" (OuterVolumeSpecName: "dshm") pod "d0a2375b-2956-4974-adaa-937c5649c3b1" (UID: "d0a2375b-2956-4974-adaa-937c5649c3b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:45.742908 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.742893    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0a2375b-2956-4974-adaa-937c5649c3b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0a2375b-2956-4974-adaa-937c5649c3b1" (UID: "d0a2375b-2956-4974-adaa-937c5649c3b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:27:45.743071 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.742928    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0a2375b-2956-4974-adaa-937c5649c3b1-kube-api-access-mvd9m" (OuterVolumeSpecName: "kube-api-access-mvd9m") pod "d0a2375b-2956-4974-adaa-937c5649c3b1" (UID: "d0a2375b-2956-4974-adaa-937c5649c3b1"). InnerVolumeSpecName "kube-api-access-mvd9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:45.842338 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.842293    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a2375b-2956-4974-adaa-937c5649c3b1-tls-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:45.842338 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.842334    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvd9m\" (UniqueName: \"kubernetes.io/projected/d0a2375b-2956-4974-adaa-937c5649c3b1-kube-api-access-mvd9m\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:45.842338 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:45.842343    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a2375b-2956-4974-adaa-937c5649c3b1-dshm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:46.071538 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.071429    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0a2375b-2956-4974-adaa-937c5649c3b1" containerID="45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8" exitCode=0
Jul 14 20:27:46.071538 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.071482    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"
Jul 14 20:27:46.071538 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.071501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b" event={"ID":"d0a2375b-2956-4974-adaa-937c5649c3b1","Type":"ContainerDied","Data":"45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8"}
Jul 14 20:27:46.071860 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.071554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b" event={"ID":"d0a2375b-2956-4974-adaa-937c5649c3b1","Type":"ContainerDied","Data":"e37ef4fa0642accd6e56943db96224fc7c0933a2a2dde2c2c2dc5b31b37c79f6"}
Jul 14 20:27:46.071860 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.071581    2581 scope.go:117] "RemoveContainer" containerID="45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8"
Jul 14 20:27:46.073544 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.073519    2581 generic.go:356] "Generic (PLEG): container finished" podID="a87c54dd-4607-43fa-8850-b16ea68502cf" containerID="47c66e796d9391bf76801080596308bb78d50e4d03f1a02ecd3ab6a8ea6b1a49" exitCode=0
Jul 14 20:27:46.073688 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.073550    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697782848" event={"ID":"a87c54dd-4607-43fa-8850-b16ea68502cf","Type":"ContainerDied","Data":"47c66e796d9391bf76801080596308bb78d50e4d03f1a02ecd3ab6a8ea6b1a49"}
Jul 14 20:27:46.082513 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.082498    2581 scope.go:117] "RemoveContainer" containerID="45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8"
Jul 14 20:27:46.082781 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:27:46.082761    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8\": container with ID starting with 45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8 not found: ID does not exist" containerID="45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8"
Jul 14 20:27:46.082830 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.082792    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8"} err="failed to get container status \"45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8\": rpc error: code = NotFound desc = could not find container \"45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8\": container with ID starting with 45b36c92cf333ef05155ebe4ae886f46222dac2116fcbd076c44cfebdb12a1c8 not found: ID does not exist"
Jul 14 20:27:46.107272 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.107235    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"]
Jul 14 20:27:46.111748 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.111717    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-76db83/rate-test-model-76db83-kserve-8f44fcd99-fzl4b"]
Jul 14 20:27:46.270426 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.270366    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg" podUID="ca0f0036-19a3-4250-918b-fb9d4033d31c" containerName="main" containerID="cri-o://7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30" gracePeriod=45
Jul 14 20:27:46.547712 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.547681    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:27:46.549712 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.549690    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-dshm\") pod \"ca0f0036-19a3-4250-918b-fb9d4033d31c\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") "
Jul 14 20:27:46.549776 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.549749    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-home\") pod \"ca0f0036-19a3-4250-918b-fb9d4033d31c\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") "
Jul 14 20:27:46.549776 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.549770    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwp8w\" (UniqueName: \"kubernetes.io/projected/ca0f0036-19a3-4250-918b-fb9d4033d31c-kube-api-access-dwp8w\") pod \"ca0f0036-19a3-4250-918b-fb9d4033d31c\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") "
Jul 14 20:27:46.549865 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.549808    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-tmp-dir\") pod \"ca0f0036-19a3-4250-918b-fb9d4033d31c\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") "
Jul 14 20:27:46.549865 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.549825    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-model-cache\") pod \"ca0f0036-19a3-4250-918b-fb9d4033d31c\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") "
Jul 14 20:27:46.549952 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.549881    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca0f0036-19a3-4250-918b-fb9d4033d31c-tls-certs\") pod \"ca0f0036-19a3-4250-918b-fb9d4033d31c\" (UID: \"ca0f0036-19a3-4250-918b-fb9d4033d31c\") "
Jul 14 20:27:46.550077 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.550028    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-home" (OuterVolumeSpecName: "home") pod "ca0f0036-19a3-4250-918b-fb9d4033d31c" (UID: "ca0f0036-19a3-4250-918b-fb9d4033d31c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:46.550157 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.550104    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca0f0036-19a3-4250-918b-fb9d4033d31c" (UID: "ca0f0036-19a3-4250-918b-fb9d4033d31c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:46.550219 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.550163    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca0f0036-19a3-4250-918b-fb9d4033d31c" (UID: "ca0f0036-19a3-4250-918b-fb9d4033d31c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:46.550312 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.550295    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-home\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:46.550362 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.550318    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-tmp-dir\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:46.550362 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.550332    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-model-cache\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:46.552079 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.552055    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-dshm" (OuterVolumeSpecName: "dshm") pod "ca0f0036-19a3-4250-918b-fb9d4033d31c" (UID: "ca0f0036-19a3-4250-918b-fb9d4033d31c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 20:27:46.552079 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.552060    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca0f0036-19a3-4250-918b-fb9d4033d31c-kube-api-access-dwp8w" (OuterVolumeSpecName: "kube-api-access-dwp8w") pod "ca0f0036-19a3-4250-918b-fb9d4033d31c" (UID: "ca0f0036-19a3-4250-918b-fb9d4033d31c"). InnerVolumeSpecName "kube-api-access-dwp8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:46.552202 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.552058    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca0f0036-19a3-4250-918b-fb9d4033d31c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca0f0036-19a3-4250-918b-fb9d4033d31c" (UID: "ca0f0036-19a3-4250-918b-fb9d4033d31c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:27:46.651210 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.651117    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwp8w\" (UniqueName: \"kubernetes.io/projected/ca0f0036-19a3-4250-918b-fb9d4033d31c-kube-api-access-dwp8w\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:46.651210 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.651152    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca0f0036-19a3-4250-918b-fb9d4033d31c-tls-certs\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:46.651210 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:46.651162    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca0f0036-19a3-4250-918b-fb9d4033d31c-dshm\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:47.081339 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.081306    2581 generic.go:356] "Generic (PLEG): container finished" podID="ca0f0036-19a3-4250-918b-fb9d4033d31c" containerID="7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30" exitCode=0
Jul 14 20:27:47.081528 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.081354    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"
Jul 14 20:27:47.081528 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.081387    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg" event={"ID":"ca0f0036-19a3-4250-918b-fb9d4033d31c","Type":"ContainerDied","Data":"7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30"}
Jul 14 20:27:47.081528 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.081450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg" event={"ID":"ca0f0036-19a3-4250-918b-fb9d4033d31c","Type":"ContainerDied","Data":"d2852fece8aff7ee6359ba484bb15b4e95ebaca179c86f937569d9a1d794ca71"}
Jul 14 20:27:47.081528 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.081469    2581 scope.go:117] "RemoveContainer" containerID="7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30"
Jul 14 20:27:47.092666 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.092640    2581 scope.go:117] "RemoveContainer" containerID="7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30"
Jul 14 20:27:47.093010 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:27:47.092991    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30\": container with ID starting with 7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30 not found: ID does not exist" containerID="7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30"
Jul 14 20:27:47.093055 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.093020    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30"} err="failed to get container status \"7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30\": rpc error: code = NotFound desc = could not find container \"7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30\": container with ID starting with 7784adc26bf91348aef53a72dc0ff6c781dc651b2c1966015db0de127f00fa30 not found: ID does not exist"
Jul 14 20:27:47.105567 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.105534    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"]
Jul 14 20:27:47.109440 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.109414    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ecc78d/rate-test-model-ecc78d-kserve-79f7744b99-f69dg"]
Jul 14 20:27:47.224011 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.223980    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:47.244981 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.244957    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-139758697782848_a87c54dd-4607-43fa-8850-b16ea68502cf/test-curl-7491-139758697782848/0.log"
Jul 14 20:27:47.256699 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.256666    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sncnc\" (UniqueName: \"kubernetes.io/projected/a87c54dd-4607-43fa-8850-b16ea68502cf-kube-api-access-sncnc\") pod \"a87c54dd-4607-43fa-8850-b16ea68502cf\" (UID: \"a87c54dd-4607-43fa-8850-b16ea68502cf\") "
Jul 14 20:27:47.258698 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.258677    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a87c54dd-4607-43fa-8850-b16ea68502cf-kube-api-access-sncnc" (OuterVolumeSpecName: "kube-api-access-sncnc") pod "a87c54dd-4607-43fa-8850-b16ea68502cf" (UID: "a87c54dd-4607-43fa-8850-b16ea68502cf"). InnerVolumeSpecName "kube-api-access-sncnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:47.282378 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.282014    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:47.285500 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.285469    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:47.338829 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.338736    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a87c54dd-4607-43fa-8850-b16ea68502cf" path="/var/lib/kubelet/pods/a87c54dd-4607-43fa-8850-b16ea68502cf/volumes"
Jul 14 20:27:47.339197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.339183    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca0f0036-19a3-4250-918b-fb9d4033d31c" path="/var/lib/kubelet/pods/ca0f0036-19a3-4250-918b-fb9d4033d31c/volumes"
Jul 14 20:27:47.339568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.339556    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0a2375b-2956-4974-adaa-937c5649c3b1" path="/var/lib/kubelet/pods/d0a2375b-2956-4974-adaa-937c5649c3b1/volumes"
Jul 14 20:27:47.357671 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.357633    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sncnc\" (UniqueName: \"kubernetes.io/projected/a87c54dd-4607-43fa-8850-b16ea68502cf-kube-api-access-sncnc\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:47.409205 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409162    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:47.409656 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409641    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0a2375b-2956-4974-adaa-937c5649c3b1" containerName="main"
Jul 14 20:27:47.409710 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409658    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0a2375b-2956-4974-adaa-937c5649c3b1" containerName="main"
Jul 14 20:27:47.409710 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409667    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca0f0036-19a3-4250-918b-fb9d4033d31c" containerName="main"
Jul 14 20:27:47.409710 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409672    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca0f0036-19a3-4250-918b-fb9d4033d31c" containerName="main"
Jul 14 20:27:47.409710 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409683    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87c54dd-4607-43fa-8850-b16ea68502cf" containerName="test-curl-7491-139758697782848"
Jul 14 20:27:47.409710 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409690    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87c54dd-4607-43fa-8850-b16ea68502cf" containerName="test-curl-7491-139758697782848"
Jul 14 20:27:47.409888 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409756    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0a2375b-2956-4974-adaa-937c5649c3b1" containerName="main"
Jul 14 20:27:47.409888 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409764    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87c54dd-4607-43fa-8850-b16ea68502cf" containerName="test-curl-7491-139758697782848"
Jul 14 20:27:47.409888 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.409773    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca0f0036-19a3-4250-918b-fb9d4033d31c" containerName="main"
Jul 14 20:27:47.414263 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.414237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:47.416377 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.416348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:47.458568 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.458516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csrm7\" (UniqueName: \"kubernetes.io/projected/256ebe5b-4843-408c-9684-b591f4a03f4c-kube-api-access-csrm7\") pod \"test-curl-7491-139758697782848\" (UID: \"256ebe5b-4843-408c-9684-b591f4a03f4c\") " pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:47.559126 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.559080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csrm7\" (UniqueName: \"kubernetes.io/projected/256ebe5b-4843-408c-9684-b591f4a03f4c-kube-api-access-csrm7\") pod \"test-curl-7491-139758697782848\" (UID: \"256ebe5b-4843-408c-9684-b591f4a03f4c\") " pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:47.568085 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.568053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csrm7\" (UniqueName: \"kubernetes.io/projected/256ebe5b-4843-408c-9684-b591f4a03f4c-kube-api-access-csrm7\") pod \"test-curl-7491-139758697782848\" (UID: \"256ebe5b-4843-408c-9684-b591f4a03f4c\") " pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:47.724696 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.724655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:47.872564 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:47.872535    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:47.874563 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:27:47.874533    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod256ebe5b_4843_408c_9684_b591f4a03f4c.slice/crio-a8ce452dca8995e9f14b393809abc91cb3b43c9622c48a1abeba81bead6937da WatchSource:0}: Error finding container a8ce452dca8995e9f14b393809abc91cb3b43c9622c48a1abeba81bead6937da: Status 404 returned error can't find the container with id a8ce452dca8995e9f14b393809abc91cb3b43c9622c48a1abeba81bead6937da
Jul 14 20:27:48.092347 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:48.092241    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:48.092347 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:48.092246    2581 scope.go:117] "RemoveContainer" containerID="47c66e796d9391bf76801080596308bb78d50e4d03f1a02ecd3ab6a8ea6b1a49"
Jul 14 20:27:48.093712 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:48.093679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697782848" event={"ID":"256ebe5b-4843-408c-9684-b591f4a03f4c","Type":"ContainerStarted","Data":"a8ce452dca8995e9f14b393809abc91cb3b43c9622c48a1abeba81bead6937da"}
Jul 14 20:27:49.104197 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:49.104155    2581 generic.go:356] "Generic (PLEG): container finished" podID="256ebe5b-4843-408c-9684-b591f4a03f4c" containerID="b6a9593f5f638751a963ced715dbd084b98729a6f2922acb9257d4456cff08d8" exitCode=0
Jul 14 20:27:49.104602 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:49.104241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-139758697782848" event={"ID":"256ebe5b-4843-408c-9684-b591f4a03f4c","Type":"ContainerDied","Data":"b6a9593f5f638751a963ced715dbd084b98729a6f2922acb9257d4456cff08d8"}
Jul 14 20:27:50.271559 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.271531    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:50.281751 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.281719    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csrm7\" (UniqueName: \"kubernetes.io/projected/256ebe5b-4843-408c-9684-b591f4a03f4c-kube-api-access-csrm7\") pod \"256ebe5b-4843-408c-9684-b591f4a03f4c\" (UID: \"256ebe5b-4843-408c-9684-b591f4a03f4c\") "
Jul 14 20:27:50.283828 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.283796    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/256ebe5b-4843-408c-9684-b591f4a03f4c-kube-api-access-csrm7" (OuterVolumeSpecName: "kube-api-access-csrm7") pod "256ebe5b-4843-408c-9684-b591f4a03f4c" (UID: "256ebe5b-4843-408c-9684-b591f4a03f4c"). InnerVolumeSpecName "kube-api-access-csrm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:27:50.294503 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.294466    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-139758697782848_256ebe5b-4843-408c-9684-b591f4a03f4c/test-curl-7491-139758697782848/0.log"
Jul 14 20:27:50.323413 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.323353    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:50.328049 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.328012    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-139758697782848"]
Jul 14 20:27:50.382961 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.382870    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csrm7\" (UniqueName: \"kubernetes.io/projected/256ebe5b-4843-408c-9684-b591f4a03f4c-kube-api-access-csrm7\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:50.520901 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.520861    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"]
Jul 14 20:27:50.521431 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.521382    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="256ebe5b-4843-408c-9684-b591f4a03f4c" containerName="test-curl-7491-139758697782848"
Jul 14 20:27:50.521431 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.521418    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="256ebe5b-4843-408c-9684-b591f4a03f4c" containerName="test-curl-7491-139758697782848"
Jul 14 20:27:50.521612 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.521522    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="256ebe5b-4843-408c-9684-b591f4a03f4c" containerName="test-curl-7491-139758697782848"
Jul 14 20:27:50.524752 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.524727    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"
Jul 14 20:27:50.527248 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.527226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:27:50.533107 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.533074    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"]
Jul 14 20:27:50.584116 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.584074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8156e88-61d6-4f41-98e9-3b1a16b4dfc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6\" (UID: \"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"
Jul 14 20:27:50.685235 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.685133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8156e88-61d6-4f41-98e9-3b1a16b4dfc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6\" (UID: \"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"
Jul 14 20:27:50.685854 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.685832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8156e88-61d6-4f41-98e9-3b1a16b4dfc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6\" (UID: \"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"
Jul 14 20:27:50.837854 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.837806    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"
Jul 14 20:27:50.992612 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:50.992584    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"]
Jul 14 20:27:50.993882 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:27:50.993858    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8156e88_61d6_4f41_98e9_3b1a16b4dfc0.slice/crio-eb45897bf0e531251d728cef973e9e87d8a05f4a92fb6b1fda377e6909349a94 WatchSource:0}: Error finding container eb45897bf0e531251d728cef973e9e87d8a05f4a92fb6b1fda377e6909349a94: Status 404 returned error can't find the container with id eb45897bf0e531251d728cef973e9e87d8a05f4a92fb6b1fda377e6909349a94
Jul 14 20:27:51.119773 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:51.119723    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6" event={"ID":"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0","Type":"ContainerStarted","Data":"eb45897bf0e531251d728cef973e9e87d8a05f4a92fb6b1fda377e6909349a94"}
Jul 14 20:27:51.121038 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:51.121020    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:51.121153 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:51.121060    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8ce452dca8995e9f14b393809abc91cb3b43c9622c48a1abeba81bead6937da"
Jul 14 20:27:51.139094 ip-10-0-130-104 kubenswrapper[2581]: E0714 20:27:51.139056    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-139758697782848\" is forbidden: User \"system:node:ip-10-0-130-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-104.ec2.internal' and this object" podUID="256ebe5b-4843-408c-9684-b591f4a03f4c" pod="opendatahub/test-curl-7491-139758697782848"
Jul 14 20:27:51.339468 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:51.339428    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="256ebe5b-4843-408c-9684-b591f4a03f4c" path="/var/lib/kubelet/pods/256ebe5b-4843-408c-9684-b591f4a03f4c/volumes"
Jul 14 20:27:52.129147 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:52.129046    2581 generic.go:356] "Generic (PLEG): container finished" podID="e8156e88-61d6-4f41-98e9-3b1a16b4dfc0" containerID="d56f3a4f8ee17af379c6c58bba19caa74669c53355b5f952eca35bdc5263aa48" exitCode=0
Jul 14 20:27:52.129147 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:52.129116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6" event={"ID":"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0","Type":"ContainerDied","Data":"d56f3a4f8ee17af379c6c58bba19caa74669c53355b5f952eca35bdc5263aa48"}
Jul 14 20:27:53.295003 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:53.294971    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"
Jul 14 20:27:53.307814 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:53.307779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8156e88-61d6-4f41-98e9-3b1a16b4dfc0-maas-api-service-ca\") pod \"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0\" (UID: \"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0\") "
Jul 14 20:27:53.308190 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:53.308168    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8156e88-61d6-4f41-98e9-3b1a16b4dfc0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8156e88-61d6-4f41-98e9-3b1a16b4dfc0" (UID: "e8156e88-61d6-4f41-98e9-3b1a16b4dfc0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:27:53.408685 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:53.408588    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8156e88-61d6-4f41-98e9-3b1a16b4dfc0-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:27:54.143613 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:54.143582    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6"
Jul 14 20:27:54.143831 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:54.143626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6" event={"ID":"e8156e88-61d6-4f41-98e9-3b1a16b4dfc0","Type":"ContainerDied","Data":"eb45897bf0e531251d728cef973e9e87d8a05f4a92fb6b1fda377e6909349a94"}
Jul 14 20:27:54.143831 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:27:54.143685    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb45897bf0e531251d728cef973e9e87d8a05f4a92fb6b1fda377e6909349a94"
Jul 14 20:28:04.835546 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:04.835506    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:28:16.729712 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.729672    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"]
Jul 14 20:28:16.730212 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.730197    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8156e88-61d6-4f41-98e9-3b1a16b4dfc0" containerName="revoke-keys"
Jul 14 20:28:16.730212 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.730210    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8156e88-61d6-4f41-98e9-3b1a16b4dfc0" containerName="revoke-keys"
Jul 14 20:28:16.730293 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.730285    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8156e88-61d6-4f41-98e9-3b1a16b4dfc0" containerName="revoke-keys"
Jul 14 20:28:16.734988 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.734966    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"
Jul 14 20:28:16.737986 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.737956    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5wdd6\""
Jul 14 20:28:16.740980 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.740952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"]
Jul 14 20:28:16.829096 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.829063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a812b9f0-648c-486b-af23-f4129aadc0f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm\" (UID: \"a812b9f0-648c-486b-af23-f4129aadc0f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"
Jul 14 20:28:16.930344 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.930294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a812b9f0-648c-486b-af23-f4129aadc0f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm\" (UID: \"a812b9f0-648c-486b-af23-f4129aadc0f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"
Jul 14 20:28:16.931093 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:16.931056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a812b9f0-648c-486b-af23-f4129aadc0f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm\" (UID: \"a812b9f0-648c-486b-af23-f4129aadc0f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"
Jul 14 20:28:17.049753 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:17.049647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"
Jul 14 20:28:17.229725 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:17.229694    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"]
Jul 14 20:28:17.231255 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:28:17.231228    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda812b9f0_648c_486b_af23_f4129aadc0f6.slice/crio-2f5c87ade3fe268037eebf20788284cf68f8870994edf66a6315e04df3a77be9 WatchSource:0}: Error finding container 2f5c87ade3fe268037eebf20788284cf68f8870994edf66a6315e04df3a77be9: Status 404 returned error can't find the container with id 2f5c87ade3fe268037eebf20788284cf68f8870994edf66a6315e04df3a77be9
Jul 14 20:28:17.298658 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:17.298616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm" event={"ID":"a812b9f0-648c-486b-af23-f4129aadc0f6","Type":"ContainerStarted","Data":"2f5c87ade3fe268037eebf20788284cf68f8870994edf66a6315e04df3a77be9"}
Jul 14 20:28:18.307209 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:18.307173    2581 generic.go:356] "Generic (PLEG): container finished" podID="a812b9f0-648c-486b-af23-f4129aadc0f6" containerID="bb381a2501bef4e3d5d5e766fe8422971dfc3b01f3e56e58dd25773bf7ae7dd6" exitCode=0
Jul 14 20:28:18.307628 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:18.307267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm" event={"ID":"a812b9f0-648c-486b-af23-f4129aadc0f6","Type":"ContainerDied","Data":"bb381a2501bef4e3d5d5e766fe8422971dfc3b01f3e56e58dd25773bf7ae7dd6"}
Jul 14 20:28:19.455643 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:19.455615    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"
Jul 14 20:28:19.559385 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:19.559341    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a812b9f0-648c-486b-af23-f4129aadc0f6-maas-api-service-ca\") pod \"a812b9f0-648c-486b-af23-f4129aadc0f6\" (UID: \"a812b9f0-648c-486b-af23-f4129aadc0f6\") "
Jul 14 20:28:19.559739 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:19.559718    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a812b9f0-648c-486b-af23-f4129aadc0f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a812b9f0-648c-486b-af23-f4129aadc0f6" (UID: "a812b9f0-648c-486b-af23-f4129aadc0f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:28:19.661022 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:19.660925    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a812b9f0-648c-486b-af23-f4129aadc0f6-maas-api-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:28:20.321230 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:20.321199    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm"
Jul 14 20:28:20.321392 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:20.321237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm" event={"ID":"a812b9f0-648c-486b-af23-f4129aadc0f6","Type":"ContainerDied","Data":"2f5c87ade3fe268037eebf20788284cf68f8870994edf66a6315e04df3a77be9"}
Jul 14 20:28:20.321392 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:20.321291    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f5c87ade3fe268037eebf20788284cf68f8870994edf66a6315e04df3a77be9"
Jul 14 20:28:23.555419 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:23.555371    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:28:23.574425 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:23.574370    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:28:27.174806 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.174758    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"]
Jul 14 20:28:27.175342 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.175045    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs" podUID="e9976699-615f-4a3c-b44b-59365c6fbb53" containerName="maas-api" containerID="cri-o://dee2c50a9cbb36e16c5a4c414fccd30ad05978e65792469e8bec19b4b5ef70b8" gracePeriod=30
Jul 14 20:28:27.374464 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.374422    2581 generic.go:356] "Generic (PLEG): container finished" podID="e9976699-615f-4a3c-b44b-59365c6fbb53" containerID="dee2c50a9cbb36e16c5a4c414fccd30ad05978e65792469e8bec19b4b5ef70b8" exitCode=0
Jul 14 20:28:27.374641 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.374452    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs" event={"ID":"e9976699-615f-4a3c-b44b-59365c6fbb53","Type":"ContainerDied","Data":"dee2c50a9cbb36e16c5a4c414fccd30ad05978e65792469e8bec19b4b5ef70b8"}
Jul 14 20:28:27.442528 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.442496    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:28:27.538745 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.538710    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9976699-615f-4a3c-b44b-59365c6fbb53-maas-api-tls\") pod \"e9976699-615f-4a3c-b44b-59365c6fbb53\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") "
Jul 14 20:28:27.538923 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.538790    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9976699-615f-4a3c-b44b-59365c6fbb53-openshift-service-ca\") pod \"e9976699-615f-4a3c-b44b-59365c6fbb53\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") "
Jul 14 20:28:27.538923 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.538911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfzdl\" (UniqueName: \"kubernetes.io/projected/e9976699-615f-4a3c-b44b-59365c6fbb53-kube-api-access-tfzdl\") pod \"e9976699-615f-4a3c-b44b-59365c6fbb53\" (UID: \"e9976699-615f-4a3c-b44b-59365c6fbb53\") "
Jul 14 20:28:27.539129 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.539105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9976699-615f-4a3c-b44b-59365c6fbb53-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9976699-615f-4a3c-b44b-59365c6fbb53" (UID: "e9976699-615f-4a3c-b44b-59365c6fbb53"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 20:28:27.539249 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.539228    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9976699-615f-4a3c-b44b-59365c6fbb53-openshift-service-ca\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:28:27.540957 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.540927    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9976699-615f-4a3c-b44b-59365c6fbb53-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9976699-615f-4a3c-b44b-59365c6fbb53" (UID: "e9976699-615f-4a3c-b44b-59365c6fbb53"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 20:28:27.541049 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.540968    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9976699-615f-4a3c-b44b-59365c6fbb53-kube-api-access-tfzdl" (OuterVolumeSpecName: "kube-api-access-tfzdl") pod "e9976699-615f-4a3c-b44b-59365c6fbb53" (UID: "e9976699-615f-4a3c-b44b-59365c6fbb53"). InnerVolumeSpecName "kube-api-access-tfzdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 20:28:27.639817 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.639776    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfzdl\" (UniqueName: \"kubernetes.io/projected/e9976699-615f-4a3c-b44b-59365c6fbb53-kube-api-access-tfzdl\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:28:27.639817 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:27.639812    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9976699-615f-4a3c-b44b-59365c6fbb53-maas-api-tls\") on node \"ip-10-0-130-104.ec2.internal\" DevicePath \"\""
Jul 14 20:28:28.383727 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:28.383697    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"
Jul 14 20:28:28.384203 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:28.383739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs" event={"ID":"e9976699-615f-4a3c-b44b-59365c6fbb53","Type":"ContainerDied","Data":"56535738db73321e41c371bae93e8f3c8660e8123bdd64effd41c5f49dc56bfa"}
Jul 14 20:28:28.384203 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:28.383801    2581 scope.go:117] "RemoveContainer" containerID="dee2c50a9cbb36e16c5a4c414fccd30ad05978e65792469e8bec19b4b5ef70b8"
Jul 14 20:28:28.411074 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:28.411036    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"]
Jul 14 20:28:28.418114 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:28.418083    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecc78d-6c85d89c8f-94nzs"]
Jul 14 20:28:29.339414 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:29.339369    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9976699-615f-4a3c-b44b-59365c6fbb53" path="/var/lib/kubelet/pods/e9976699-615f-4a3c-b44b-59365c6fbb53/volumes"
Jul 14 20:28:30.434591 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:30.434555    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cj2mn"]
Jul 14 20:28:48.683176 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:48.683141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5jdhc_c5ee1074-bafe-4acf-bd7f-4235b8be00f0/manager/0.log"
Jul 14 20:28:48.815285 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:48.815254    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4bwjw_489ece5a-4af8-4670-bb4a-76119587279d/manager/0.log"
Jul 14 20:28:49.215483 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:49.215450    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-njgf6_08542d4a-edbd-4d83-8bf3-bcec89cc3876/manager/0.log"
Jul 14 20:28:49.479334 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:49.479246    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf_60bb432d-0467-4f8b-9fc7-dec8390bf954/copy-manifests/0.log"
Jul 14 20:28:49.485990 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:49.485960    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf_60bb432d-0467-4f8b-9fc7-dec8390bf954/manager/0.log"
Jul 14 20:28:50.166265 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:50.166231    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734320-sr666_747d85e1-bacb-4702-b509-c3aa03de87dc/cleanup/0.log"
Jul 14 20:28:50.291287 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:50.291257    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734335-vz9n4_d7555b08-4c25-4de4-b270-6dcab0971c05/cleanup/0.log"
Jul 14 20:28:50.429062 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:50.428987    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-0ac1fa23-j4dwp_acf4c3d5-da86-4ca5-a7c2-f9bb4a1788c3/revoke-keys/0.log"
Jul 14 20:28:50.566531 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:50.566501    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-ffd1ab16-2cqxc_dbb4aa24-aeed-4145-bd9b-62da3c95993c/revoke-keys/0.log"
Jul 14 20:28:50.693648 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:50.693560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-c73bc1-n4r6f_296d2ea8-f18e-4e19-a0e6-6e7580f819cc/revoke-keys/0.log"
Jul 14 20:28:50.815569 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:50.815537    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-6e455f-c8r69_7b25e533-a235-48de-93e8-beca086ec6e7/revoke-keys/0.log"
Jul 14 20:28:50.933238 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:50.933210    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-8983c49d-cslh8_d89b4368-e7d5-49d1-8817-1112924b6efe/revoke-keys/0.log"
Jul 14 20:28:51.060679 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:51.060648    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-a8298f-h9l2s_07e3d2b5-ed17-414a-8e3d-ea269ba95e4c/revoke-keys/0.log"
Jul 14 20:28:51.179732 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:51.179704    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-3195eb-5l7dt_5635d2f3-e515-4220-ae4a-7e69f8950c83/revoke-keys/0.log"
Jul 14 20:28:51.295813 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:51.295776    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-dec8aa7c-jqdjh_ca44855d-0bbb-4fa9-9cb2-2247f7ff8416/revoke-keys/0.log"
Jul 14 20:28:51.415663 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:51.415581    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-43646543-h42dq_4f5a9a1a-1572-4ca8-a82e-0e40da19a9f5/revoke-keys/0.log"
Jul 14 20:28:51.528181 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:51.528149    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-6023fdbc-pr9dm_89cc34a9-ea2b-4ce2-a1f6-37d88fbf90bf/revoke-keys/0.log"
Jul 14 20:28:51.639696 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:51.639670    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-76db83-fl2w6_e8156e88-61d6-4f41-98e9-3b1a16b4dfc0/revoke-keys/0.log"
Jul 14 20:28:51.757052 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:51.757021    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-ecc78d-j5ncm_a812b9f0-648c-486b-af23-f4129aadc0f6/revoke-keys/0.log"
Jul 14 20:28:52.751030 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:52.750993    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n_39a5dc7e-5391-4e3a-a123-fded563dc644/extract/0.log"
Jul 14 20:28:52.757774 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:52.757748    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n_39a5dc7e-5391-4e3a-a123-fded563dc644/util/0.log"
Jul 14 20:28:52.765096 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:52.765066    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n_39a5dc7e-5391-4e3a-a123-fded563dc644/pull/0.log"
Jul 14 20:28:52.881454 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:52.881425    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h_138f51aa-632d-4ac7-81a8-b48881da21b7/extract/0.log"
Jul 14 20:28:52.891344 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:52.891321    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h_138f51aa-632d-4ac7-81a8-b48881da21b7/util/0.log"
Jul 14 20:28:52.898063 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:52.898039    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h_138f51aa-632d-4ac7-81a8-b48881da21b7/pull/0.log"
Jul 14 20:28:53.018195 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.018111    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq_83503fe0-1721-43f3-862f-587585d75e50/util/0.log"
Jul 14 20:28:53.026201 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.026167    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq_83503fe0-1721-43f3-862f-587585d75e50/pull/0.log"
Jul 14 20:28:53.033490 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.033467    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq_83503fe0-1721-43f3-862f-587585d75e50/extract/0.log"
Jul 14 20:28:53.161369 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.161340    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4_245dd48b-7305-4f00-af93-145b077101a3/util/0.log"
Jul 14 20:28:53.173158 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.173125    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4_245dd48b-7305-4f00-af93-145b077101a3/pull/0.log"
Jul 14 20:28:53.180136 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.180112    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4_245dd48b-7305-4f00-af93-145b077101a3/extract/0.log"
Jul 14 20:28:53.454077 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.454045    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5tcss_7366817a-8439-43ca-b6d5-a5b8dfae5437/manager/0.log"
Jul 14 20:28:53.585247 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.585218    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wchgh_741366f7-0b23-4447-b9da-9ce6f424b92c/manager/0.log"
Jul 14 20:28:53.845667 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.845580    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-nq2sf_1ebbdc46-3c55-4513-ae2b-d11867280462/registry-server/0.log"
Jul 14 20:28:53.977845 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:53.977808    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xb5rh_62502f27-1433-4130-bc31-f718ab3ceab2/manager/0.log"
Jul 14 20:28:54.102252 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:54.102166    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cj2mn_709fb926-6a75-49ca-ba6f-38fca2f82557/limitador/0.log"
Jul 14 20:28:54.227358 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:54.227322    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-r7s6h_4c26d98e-8629-4927-957d-a4cf3d43a64f/manager/0.log"
Jul 14 20:28:54.615951 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:54.615921    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45jd7s_28fd6f37-f91f-4899-b011-da9c60ca0875/istio-proxy/0.log"
Jul 14 20:28:55.003979 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:55.003949    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-w89dt_dd98c15a-ae0e-4c56-82e2-482bafa874a3/kube-auth-proxy/0.log"
Jul 14 20:28:55.138851 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:28:55.138822    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thcwq_18ad6f09-f21e-4cb1-b810-a03ca4a4b180/istio-proxy/1.log"
Jul 14 20:29:01.159867 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.159828    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7kv5j/must-gather-qgsnc"]
Jul 14 20:29:01.160326 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.160288    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a812b9f0-648c-486b-af23-f4129aadc0f6" containerName="revoke-keys"
Jul 14 20:29:01.160326 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.160299    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a812b9f0-648c-486b-af23-f4129aadc0f6" containerName="revoke-keys"
Jul 14 20:29:01.160326 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.160320    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9976699-615f-4a3c-b44b-59365c6fbb53" containerName="maas-api"
Jul 14 20:29:01.160326 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.160325    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9976699-615f-4a3c-b44b-59365c6fbb53" containerName="maas-api"
Jul 14 20:29:01.160508 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.160383    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9976699-615f-4a3c-b44b-59365c6fbb53" containerName="maas-api"
Jul 14 20:29:01.160508 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.160393    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a812b9f0-648c-486b-af23-f4129aadc0f6" containerName="revoke-keys"
Jul 14 20:29:01.163850 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.163818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.166492 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.166461    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7kv5j\"/\"kube-root-ca.crt\""
Jul 14 20:29:01.166492 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.166486    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7kv5j\"/\"default-dockercfg-7vshp\""
Jul 14 20:29:01.167202 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.167187    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7kv5j\"/\"openshift-service-ca.crt\""
Jul 14 20:29:01.172861 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.172835    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7kv5j/must-gather-qgsnc"]
Jul 14 20:29:01.267245 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.267206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqngh\" (UniqueName: \"kubernetes.io/projected/8fa20080-3e2a-446b-9669-57317b38b0f7-kube-api-access-bqngh\") pod \"must-gather-qgsnc\" (UID: \"8fa20080-3e2a-446b-9669-57317b38b0f7\") " pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.267245 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.267257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8fa20080-3e2a-446b-9669-57317b38b0f7-must-gather-output\") pod \"must-gather-qgsnc\" (UID: \"8fa20080-3e2a-446b-9669-57317b38b0f7\") " pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.367909 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.367872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8fa20080-3e2a-446b-9669-57317b38b0f7-must-gather-output\") pod \"must-gather-qgsnc\" (UID: \"8fa20080-3e2a-446b-9669-57317b38b0f7\") " pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.368141 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.367987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqngh\" (UniqueName: \"kubernetes.io/projected/8fa20080-3e2a-446b-9669-57317b38b0f7-kube-api-access-bqngh\") pod \"must-gather-qgsnc\" (UID: \"8fa20080-3e2a-446b-9669-57317b38b0f7\") " pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.368329 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.368303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8fa20080-3e2a-446b-9669-57317b38b0f7-must-gather-output\") pod \"must-gather-qgsnc\" (UID: \"8fa20080-3e2a-446b-9669-57317b38b0f7\") " pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.376722 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.376689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqngh\" (UniqueName: \"kubernetes.io/projected/8fa20080-3e2a-446b-9669-57317b38b0f7-kube-api-access-bqngh\") pod \"must-gather-qgsnc\" (UID: \"8fa20080-3e2a-446b-9669-57317b38b0f7\") " pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.474331 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.474289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7kv5j/must-gather-qgsnc"
Jul 14 20:29:01.619173 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:01.619144    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7kv5j/must-gather-qgsnc"]
Jul 14 20:29:01.621931 ip-10-0-130-104 kubenswrapper[2581]: W0714 20:29:01.621885    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fa20080_3e2a_446b_9669_57317b38b0f7.slice/crio-b62c8e693a76a8ec17b2a9f4ad75b5c4bae1df8ebc3a055f1eac71ea19e2cd31 WatchSource:0}: Error finding container b62c8e693a76a8ec17b2a9f4ad75b5c4bae1df8ebc3a055f1eac71ea19e2cd31: Status 404 returned error can't find the container with id b62c8e693a76a8ec17b2a9f4ad75b5c4bae1df8ebc3a055f1eac71ea19e2cd31
Jul 14 20:29:02.632867 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:02.632830    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7kv5j/must-gather-qgsnc" event={"ID":"8fa20080-3e2a-446b-9669-57317b38b0f7","Type":"ContainerStarted","Data":"b62c8e693a76a8ec17b2a9f4ad75b5c4bae1df8ebc3a055f1eac71ea19e2cd31"}
Jul 14 20:29:03.643811 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:03.643773    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7kv5j/must-gather-qgsnc" event={"ID":"8fa20080-3e2a-446b-9669-57317b38b0f7","Type":"ContainerStarted","Data":"71000d3998213ba4c3625078a5b577e05959a0ff8aa6e28a20917b0f53a8f445"}
Jul 14 20:29:03.644315 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:03.644156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7kv5j/must-gather-qgsnc" event={"ID":"8fa20080-3e2a-446b-9669-57317b38b0f7","Type":"ContainerStarted","Data":"f4f7f53d469ae70ee8e21c1b3122ab62b30f8dc114917b6d7ac9674375f51d89"}
Jul 14 20:29:03.662093 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:03.662022    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7kv5j/must-gather-qgsnc" podStartSLOduration=1.5327468579999999 podStartE2EDuration="2.661998255s" totalImagesPullingTime="1.129251397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:29:03.658900998 +0000 UTC m=+3041.970600455" watchObservedRunningTime="2026-07-14 20:29:03.661998255 +0000 UTC m=+3041.973697712"
Jul 14 20:29:04.541684 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:04.541652    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zdsp5_c7a2b6a9-bd74-458a-88a3-b7e5570767d4/global-pull-secret-syncer/0.log"
Jul 14 20:29:04.678158 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:04.678116    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n99s2_7cc028f4-1c3d-44d7-bf9a-8bd184c4f4ec/konnectivity-agent/0.log"
Jul 14 20:29:04.728584 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:04.728552    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-104.ec2.internal_b7a9b0255c9ea7786828afe16280e2bd/haproxy/0.log"
Jul 14 20:29:09.089646 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.089136    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n_39a5dc7e-5391-4e3a-a123-fded563dc644/extract/0.log"
Jul 14 20:29:09.113225 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.113196    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n_39a5dc7e-5391-4e3a-a123-fded563dc644/util/0.log"
Jul 14 20:29:09.137792 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.137636    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dst6n_39a5dc7e-5391-4e3a-a123-fded563dc644/pull/0.log"
Jul 14 20:29:09.171543 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.171507    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h_138f51aa-632d-4ac7-81a8-b48881da21b7/extract/0.log"
Jul 14 20:29:09.198083 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.198052    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h_138f51aa-632d-4ac7-81a8-b48881da21b7/util/0.log"
Jul 14 20:29:09.226142 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.226110    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k9b6h_138f51aa-632d-4ac7-81a8-b48881da21b7/pull/0.log"
Jul 14 20:29:09.261375 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.261340    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq_83503fe0-1721-43f3-862f-587585d75e50/extract/0.log"
Jul 14 20:29:09.291817 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.291786    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq_83503fe0-1721-43f3-862f-587585d75e50/util/0.log"
Jul 14 20:29:09.322735 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.322686    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kfdtq_83503fe0-1721-43f3-862f-587585d75e50/pull/0.log"
Jul 14 20:29:09.360925 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.360840    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4_245dd48b-7305-4f00-af93-145b077101a3/extract/0.log"
Jul 14 20:29:09.390184 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.390101    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4_245dd48b-7305-4f00-af93-145b077101a3/util/0.log"
Jul 14 20:29:09.420344 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.420309    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef129pz4_245dd48b-7305-4f00-af93-145b077101a3/pull/0.log"
Jul 14 20:29:09.772895 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.772861    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5tcss_7366817a-8439-43ca-b6d5-a5b8dfae5437/manager/0.log"
Jul 14 20:29:09.808169 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.808133    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wchgh_741366f7-0b23-4447-b9da-9ce6f424b92c/manager/0.log"
Jul 14 20:29:09.898926 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:09.898894    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-nq2sf_1ebbdc46-3c55-4513-ae2b-d11867280462/registry-server/0.log"
Jul 14 20:29:10.090959 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:10.090787    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xb5rh_62502f27-1433-4130-bc31-f718ab3ceab2/manager/0.log"
Jul 14 20:29:10.118669 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:10.118634    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cj2mn_709fb926-6a75-49ca-ba6f-38fca2f82557/limitador/0.log"
Jul 14 20:29:10.214830 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:10.214773    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-r7s6h_4c26d98e-8629-4927-957d-a4cf3d43a64f/manager/0.log"
Jul 14 20:29:12.140664 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.139830    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-222tg_79b168f0-d438-4064-9ee3-97bad5dfebfd/cluster-monitoring-operator/0.log"
Jul 14 20:29:12.179516 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.179483    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vpkrs_e0870fac-f247-4234-be87-42613c05d2a2/kube-state-metrics/0.log"
Jul 14 20:29:12.212510 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.212476    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vpkrs_e0870fac-f247-4234-be87-42613c05d2a2/kube-rbac-proxy-main/0.log"
Jul 14 20:29:12.275619 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.275542    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-vpkrs_e0870fac-f247-4234-be87-42613c05d2a2/kube-rbac-proxy-self/0.log"
Jul 14 20:29:12.358628 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.358592    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64d5b566b6-v6d9q_e74e9c3f-051c-4a53-b998-e95da94f45c0/metrics-server/0.log"
Jul 14 20:29:12.602923 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.602835    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvn6s_f99cd6dd-2793-41b4-87b3-bd56e4450401/node-exporter/0.log"
Jul 14 20:29:12.642870 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.642699    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvn6s_f99cd6dd-2793-41b4-87b3-bd56e4450401/kube-rbac-proxy/0.log"
Jul 14 20:29:12.675221 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:12.675119    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvn6s_f99cd6dd-2793-41b4-87b3-bd56e4450401/init-textfile/0.log"
Jul 14 20:29:13.234049 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:13.234013    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/thanos-query/0.log"
Jul 14 20:29:13.260258 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:13.260218    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy-web/0.log"
Jul 14 20:29:13.292899 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:13.292871    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy/0.log"
Jul 14 20:29:13.318811 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:13.318754    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/prom-label-proxy/0.log"
Jul 14 20:29:13.345863 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:13.345832    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy-rules/0.log"
Jul 14 20:29:13.376852 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:13.376821    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59b9f95cf-9pfx6_0fbb8f4e-55da-4da4-a6a6-34414e099976/kube-rbac-proxy-metrics/0.log"
Jul 14 20:29:14.544464 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:14.544426    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-q68tv_de28e053-ed08-45f7-acd3-986ae8acd2fe/networking-console-plugin/0.log"
Jul 14 20:29:14.901809 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:14.901723    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"]
Jul 14 20:29:14.910848 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:14.910814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:14.914486 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:14.914453    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"]
Jul 14 20:29:15.029574 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.029525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-proc\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.030886 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.030855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-sys\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.031127 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.031110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v69w\" (UniqueName: \"kubernetes.io/projected/48c42768-3f29-41ce-a096-43219c5f9e84-kube-api-access-5v69w\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.031318 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.031299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-podres\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.031559 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.031540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-lib-modules\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.132772 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.132730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-proc\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.132803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-sys\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.132843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v69w\" (UniqueName: \"kubernetes.io/projected/48c42768-3f29-41ce-a096-43219c5f9e84-kube-api-access-5v69w\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.132874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-proc\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.132901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-podres\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.132909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-sys\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133005 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.132973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-lib-modules\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133371 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.133046    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-podres\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.133371 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.133167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48c42768-3f29-41ce-a096-43219c5f9e84-lib-modules\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.142004 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.141971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v69w\" (UniqueName: \"kubernetes.io/projected/48c42768-3f29-41ce-a096-43219c5f9e84-kube-api-access-5v69w\") pod \"perf-node-gather-daemonset-h6mw9\" (UID: \"48c42768-3f29-41ce-a096-43219c5f9e84\") " pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.228359 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.228266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.247241 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.247175    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/1.log"
Jul 14 20:29:15.256431 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.255991    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wvfcn_7216f409-33e7-434d-abf9-e8d9e953d3e2/console-operator/2.log"
Jul 14 20:29:15.443363 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.443336    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"]
Jul 14 20:29:15.773008 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.772900    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9" event={"ID":"48c42768-3f29-41ce-a096-43219c5f9e84","Type":"ContainerStarted","Data":"1d092ca0488470b4eb7ded7f2c0ba8624a62425ff9f2490141487f3d7fc3a153"}
Jul 14 20:29:15.773008 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.772976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9" event={"ID":"48c42768-3f29-41ce-a096-43219c5f9e84","Type":"ContainerStarted","Data":"cf8ab380b064bc3d5f774575b3c4f416b57d2ae25dda1b0706075bc298feb797"}
Jul 14 20:29:15.776167 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.776136    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:15.795445 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.795362    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9" podStartSLOduration=1.7953391509999999 podStartE2EDuration="1.795339151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:29:15.790867434 +0000 UTC m=+3054.102566888" watchObservedRunningTime="2026-07-14 20:29:15.795339151 +0000 UTC m=+3054.107038608"
Jul 14 20:29:15.851184 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.851152    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b54df4f4d-5xczm_7d5a6d6d-4c30-4488-aed6-1a5ad257602c/console/0.log"
Jul 14 20:29:15.887973 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:15.887943    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-dgcpn_24d3d98f-c435-40da-8683-3d266890b3ff/download-server/0.log"
Jul 14 20:29:16.508929 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:16.508900    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-n4lqq_43c66065-d7e3-4761-bd3f-248bbfb2a0ac/volume-data-source-validator/0.log"
Jul 14 20:29:17.660784 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:17.660754    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nn5jk_0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f/dns/0.log"
Jul 14 20:29:17.683803 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:17.683772    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nn5jk_0e87ee6b-a59f-41b0-b2d1-6a60fa45c81f/kube-rbac-proxy/0.log"
Jul 14 20:29:17.816002 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:17.815967    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z826t_cc6571ea-7b0d-4671-a89b-e427c40cecc4/dns-node-resolver/0.log"
Jul 14 20:29:18.383619 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:18.383584    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-676f444bdd-6wthl_dd6c5539-7faf-4701-b16b-237a56f2ef5e/registry/0.log"
Jul 14 20:29:18.401651 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:18.401583    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-676f444bdd-6wthl_dd6c5539-7faf-4701-b16b-237a56f2ef5e/registry/1.log"
Jul 14 20:29:18.428779 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:18.428748    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5p8x5_13c49190-4865-4f90-a7f6-3400303c9879/node-ca/0.log"
Jul 14 20:29:19.462798 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:19.462764    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45jd7s_28fd6f37-f91f-4899-b011-da9c60ca0875/istio-proxy/0.log"
Jul 14 20:29:19.642772 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:19.642740    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-w89dt_dd98c15a-ae0e-4c56-82e2-482bafa874a3/kube-auth-proxy/0.log"
Jul 14 20:29:19.679850 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:19.679812    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thcwq_18ad6f09-f21e-4cb1-b810-a03ca4a4b180/istio-proxy/0.log"
Jul 14 20:29:19.720971 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:19.720880    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thcwq_18ad6f09-f21e-4cb1-b810-a03ca4a4b180/istio-proxy/1.log"
Jul 14 20:29:20.584541 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:20.584509    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tpdjw_99fa2d76-352e-406b-84e4-968d0ffd15da/serve-healthcheck-canary/0.log"
Jul 14 20:29:21.220520 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:21.220453    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftrdr_6a2a1dc7-e4fb-4c41-bb67-69321e0a089b/kube-rbac-proxy/0.log"
Jul 14 20:29:21.249898 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:21.249865    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftrdr_6a2a1dc7-e4fb-4c41-bb67-69321e0a089b/exporter/0.log"
Jul 14 20:29:21.275532 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:21.275498    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftrdr_6a2a1dc7-e4fb-4c41-bb67-69321e0a089b/extractor/0.log"
Jul 14 20:29:22.799364 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:22.799333    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7kv5j/perf-node-gather-daemonset-h6mw9"
Jul 14 20:29:23.759717 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:23.759685    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5jdhc_c5ee1074-bafe-4acf-bd7f-4235b8be00f0/manager/0.log"
Jul 14 20:29:23.982097 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:23.981509    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4bwjw_489ece5a-4af8-4670-bb4a-76119587279d/manager/0.log"
Jul 14 20:29:24.224670 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:24.224444    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-njgf6_08542d4a-edbd-4d83-8bf3-bcec89cc3876/manager/0.log"
Jul 14 20:29:24.327809 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:24.327768    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf_60bb432d-0467-4f8b-9fc7-dec8390bf954/manager/0.log"
Jul 14 20:29:24.354750 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:24.354717    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fdfc6fcb8-g8ccf_60bb432d-0467-4f8b-9fc7-dec8390bf954/copy-manifests/0.log"
Jul 14 20:29:26.133745 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:26.133699    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45jd7s_28fd6f37-f91f-4899-b011-da9c60ca0875/istio-proxy/0.log"
Jul 14 20:29:26.282147 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:26.282109    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-w89dt_dd98c15a-ae0e-4c56-82e2-482bafa874a3/kube-auth-proxy/0.log"
Jul 14 20:29:26.309103 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:26.309066    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thcwq_18ad6f09-f21e-4cb1-b810-a03ca4a4b180/istio-proxy/0.log"
Jul 14 20:29:26.349423 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:26.348906    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thcwq_18ad6f09-f21e-4cb1-b810-a03ca4a4b180/istio-proxy/1.log"
Jul 14 20:29:27.658016 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:27.657987    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45jd7s_28fd6f37-f91f-4899-b011-da9c60ca0875/istio-proxy/0.log"
Jul 14 20:29:27.808267 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:27.808185    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59f575c9d9-w89dt_dd98c15a-ae0e-4c56-82e2-482bafa874a3/kube-auth-proxy/0.log"
Jul 14 20:29:27.828237 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:27.828201    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thcwq_18ad6f09-f21e-4cb1-b810-a03ca4a4b180/istio-proxy/0.log"
Jul 14 20:29:27.870750 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:27.870712    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thcwq_18ad6f09-f21e-4cb1-b810-a03ca4a4b180/istio-proxy/1.log"
Jul 14 20:29:31.217138 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:31.217095    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-qxvsw_2dfcd161-8f36-49f1-bcc0-549c01a2da6f/migrator/0.log"
Jul 14 20:29:31.241305 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:31.241272    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-qxvsw_2dfcd161-8f36-49f1-bcc0-549c01a2da6f/graceful-termination/0.log"
Jul 14 20:29:33.049114 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:33.049080    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v2sss_0777416d-4547-40b1-82f8-78911a02ad4a/kube-multus-additional-cni-plugins/0.log"
Jul 14 20:29:33.072105 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:33.072075    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v2sss_0777416d-4547-40b1-82f8-78911a02ad4a/egress-router-binary-copy/0.log"
Jul 14 20:29:33.098079 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:33.098040    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v2sss_0777416d-4547-40b1-82f8-78911a02ad4a/cni-plugins/0.log"
Jul 14 20:29:33.122571 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:33.122541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v2sss_0777416d-4547-40b1-82f8-78911a02ad4a/bond-cni-plugin/0.log"
Jul 14 20:29:33.143541 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:33.143512    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v2sss_0777416d-4547-40b1-82f8-78911a02ad4a/routeoverride-cni/0.log"
Jul 14 20:29:33.165755 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:33.165712    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v2sss_0777416d-4547-40b1-82f8-78911a02ad4a/whereabouts-cni-bincopy/0.log"
Jul 14 20:29:33.188095 ip-10-0-130-104 kubenswrapper[2581]: I0714 20:29:33.188063    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v2sss_0777416d-4547-40b1-82f8-78911a02ad4a/whereabouts-cni/0.log"