Jul 13 20:53:58.947571 ip-10-0-138-146 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 20:53:59.350176 ip-10-0-138-146 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 20:53:59.350176 ip-10-0-138-146 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 20:53:59.350176 ip-10-0-138-146 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 20:53:59.350176 ip-10-0-138-146 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 20:53:59.350176 ip-10-0-138-146 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 20:53:59.352160 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.352042    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 20:53:59.357383 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357364    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 20:53:59.357383 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357380    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 20:53:59.357383 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357384    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 20:53:59.357383 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357388    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357392    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357416    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357421    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357429    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357433    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357438    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357443    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357447    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357452    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357456    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357460    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357464    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357467    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357471    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357475    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357479    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357483    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357497    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357502    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 20:53:59.357623 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357506    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357511    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357515    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357519    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357524    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357528    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357532    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357538    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357542    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357546    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357551    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357556    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357561    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357566    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357571    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357575    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357580    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357584    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357588    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357592    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 20:53:59.358457 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357597    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357601    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357605    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357609    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357613    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357617    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357621    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357625    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357629    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357633    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357638    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357643    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357647    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357651    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357655    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357659    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357663    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357667    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357672    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357676    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 20:53:59.359201 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357680    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357684    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357689    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357693    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357699    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357705    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357709    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357714    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357719    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357723    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357732    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357738    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357744    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357749    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357754    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357758    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357763    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357768    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357773    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 20:53:59.359761 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357777    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357781    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357787    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357793    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357798    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357803    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357807    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357812    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357818    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357824    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357829    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357833    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357837    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357841    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357846    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357850    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 20:53:59.360211 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.357854    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360559    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360570    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360574    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360578    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360581    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360584    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360588    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360591    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360594    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360597    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360601    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360607    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360610    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360613    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360616    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360619    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360621    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360624    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 20:53:59.360625 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360629    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360633    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360637    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360646    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360649    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360655    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360658    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360661    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360663    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360666    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360669    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360672    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360674    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360677    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360680    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360684    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360687    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360692    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360695    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 20:53:59.361122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360698    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360701    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360704    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360707    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360710    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360713    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360716    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360719    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360722    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360725    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360730    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360732    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360735    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360738    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360741    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360744    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360747    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360755    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360758    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 20:53:59.361599 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360761    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360764    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360766    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360771    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360775    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360778    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360780    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360783    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360785    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360788    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360791    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360793    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360796    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360798    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360801    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360806    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360809    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360811    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360814    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360817    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360819    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 20:53:59.362061 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360822    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360824    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360827    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360829    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360832    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360835    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360837    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360843    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360846    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360849    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360859    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360862    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360864    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360868    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360872    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360875    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360877    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360880    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360883    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360888    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 20:53:59.362581 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360891    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.360893    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361656    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361663    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361668    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361671    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361674    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361677    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361680    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361683    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361685    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361688    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361690    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361696    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361699    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361701    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361704    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361707    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361710    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 20:53:59.363080 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361713    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361716    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361719    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361727    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361731    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361733    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361738    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361741    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361744    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361747    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361750    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361752    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361755    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361757    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361760    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361763    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361765    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361768    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361770    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361775    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 20:53:59.363564 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361778    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361780    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361783    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361786    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361789    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361792    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361794    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361797    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361799    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361802    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361804    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361809    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361812    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361815    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361818    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361820    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361830    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361833    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361835    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361838    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 20:53:59.364045 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361841    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361846    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361849    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361855    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361858    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361867    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361870    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361873    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361876    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361879    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361882    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361884    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361887    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361890    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361892    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361895    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361900    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361902    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361905    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 20:53:59.364551 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361908    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361910    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361913    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361916    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361918    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361924    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361927    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361929    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361934    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361937    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361945    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361948    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361951    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361954    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361956    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361959    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361963    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361966    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361969    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361972    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 20:53:59.365003 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361975    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361981    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.361984    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362255    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362265    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362273    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362278    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362282    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362285    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362290    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362294    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362297    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362300    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362304    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362307    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362310    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362313    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362316    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362319    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362322    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362325    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362329    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362332    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362349    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 20:53:59.365497 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362353    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362356    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362360    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362363    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362366    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362370    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362373    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362376    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362379    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362382    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362385    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362390    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362393    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362395    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362398    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362401    2556 flags.go:64] FLAG: --enable-server="true"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362404    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362413    2556 flags.go:64] FLAG: --event-burst="100"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362416    2556 flags.go:64] FLAG: --event-qps="50"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362419    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362423    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362426    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362430    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362432    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362436    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 20:53:59.366066 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362439    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362442    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362445    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362454    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362457    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362460    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362462    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362465    2556 flags.go:64] FLAG: --feature-gates=""
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362469    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362472    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362475    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362478    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362481    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362484    2556 flags.go:64] FLAG: --help="false"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362487    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362490    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362493    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362496    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362499    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362502    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362505    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362508    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362511    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 20:53:59.366711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362514    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362516    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362520    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362522    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362525    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362528    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362531    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362533    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362536    2556 flags.go:64] FLAG: --lock-file=""
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362539    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362541    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362544    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362554    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362562    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362565    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362568    2556 flags.go:64] FLAG: --logging-format="text"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362570    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362574    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362577    2556 flags.go:64] FLAG: --manifest-url=""
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362579    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362585    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362588    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362592    2556 flags.go:64] FLAG: --max-pods="110"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362595    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362598    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 20:53:59.367310 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362601    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362605    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362608    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362611    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362613    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362621    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362624    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362627    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362630    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362633    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362638    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362641    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362648    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362651    2556 flags.go:64] FLAG: --port="10250"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362653    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362657    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f9b797c6fa23aa07"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362660    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362662    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362665    2556 flags.go:64] FLAG: --register-node="true"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362668    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362672    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362680    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362683    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362686    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362690    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 20:53:59.367948 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362693    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362695    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362698    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362701    2556 flags.go:64] FLAG: --runonce="false"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362703    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362706    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362709    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362712    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362715    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362718    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362721    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362724    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362726    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362729    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362732    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362735    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362738    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362740    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362743    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362750    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362753    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362756    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362762    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362765    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362768    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 20:53:59.368574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362771    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362773    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362776    2556 flags.go:64] FLAG: --v="2"
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362780    2556 flags.go:64] FLAG: --version="false"
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362791    2556 flags.go:64] FLAG: --vmodule=""
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362795    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.362798    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362952    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362957    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362961    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362965    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362968    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362971    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362974    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362977    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362980    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362983    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362986    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362988    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362991    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362993    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 20:53:59.369179 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362996    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.362998    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363001    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363003    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363005    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363010    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363012    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363016    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363019    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363022    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363024    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363027    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363030    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363032    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363035    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363037    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363045    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363048    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363051    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363053    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 20:53:59.369737 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363056    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363058    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363062    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363065    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363068    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363070    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363073    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363075    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363078    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363081    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363083    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363086    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363089    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363091    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363094    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363096    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363099    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363102    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363105    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363107    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 20:53:59.370225 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363109    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363112    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363114    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363117    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363119    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363122    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363124    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363127    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363130    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363138    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363140    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363143    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363146    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363148    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363152    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363154    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363157    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363159    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363162    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363164    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 20:53:59.370770 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363167    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363169    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363172    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363174    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363177    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363180    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363182    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363185    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363187    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363191    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363193    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363195    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363198    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363200    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363203    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363205    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363208    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363210    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363214    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 20:53:59.371271 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363218    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363221    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363223    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363232    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363234    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.363237    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.363914    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.370458    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.370474    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.370494    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.370499    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370553    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370558    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370562    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370566    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 20:53:59.371749 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370569    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370572    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370575    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370578    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370581    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370583    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370586    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370589    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370591    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370594    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370596    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370599    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370601    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370604    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370606    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370609    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370611    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370614    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370616    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 20:53:59.372122 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370620    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370627    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370630    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370633    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370636    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370640    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370645    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370648    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370650    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370654    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370657    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370660    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370663    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370666    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370668    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370671    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370673    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370676    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370679    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 20:53:59.372592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370681    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370684    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370686    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370689    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370692    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370694    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370697    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370699    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370702    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370704    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370707    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370709    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370712    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370714    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370717    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370720    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370723    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370725    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370728    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370731    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 20:53:59.373042 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370733    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370736    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370739    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370741    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370744    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370746    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370749    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370751    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370754    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370757    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370759    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370762    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370764    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370767    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370769    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370772    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370775    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370777    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370780    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370783    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 20:53:59.373556 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370786    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370788    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370791    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370793    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370796    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370799    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370801    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370803    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370806    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370809    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370812    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370815    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370817    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370820    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370822    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370825    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 20:53:59.374032 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370827    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.370832    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370962    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370967    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370969    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370972    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370975    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370977    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370980    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370982    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370986    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370989    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370992    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370995    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 20:53:59.374444 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.370997    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371000    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371002    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371005    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371007    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371010    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371012    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371014    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371017    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371019    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371022    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371025    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371029    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371033    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371036    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371039    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371041    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371044    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371047    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 20:53:59.374795 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371049    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371052    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371055    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371058    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371060    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371063    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371065    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371068    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371070    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371073    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371075    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371078    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371080    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371083    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371085    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371088    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371090    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371092    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371095    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371097    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 20:53:59.375262 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371100    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371103    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371105    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371107    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371110    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371113    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371116    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371118    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371121    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371123    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371126    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371128    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371131    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371134    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371136    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371139    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371141    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371143    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371146    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371149    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 20:53:59.375764 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371151    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371153    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371156    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371158    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371161    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371163    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371166    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371169    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371176    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371178    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371181    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371183    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371186    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371188    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371191    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371193    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371196    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371199    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371202    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371204    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 20:53:59.376244 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371207    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371210    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371213    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371216    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371218    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371221    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371223    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:53:59.371225    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.371230    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.371349    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.373387    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.374863    2556 server.go:1019] "Starting client certificate rotation"
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.374977    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 20:53:59.376778 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.375010    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 20:53:59.377455 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.377443    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 20:53:59.378755 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.378737    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 20:53:59.394260 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.394239    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 13 20:53:59.396050 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.396040    2556 log.go:25] "Validated CRI v1 image API"
Jul 13 20:53:59.396778 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.396763    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 20:53:59.398965 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.398946    2556 fs.go:135] Filesystem UUIDs: map[3cfe968b-f9c2-46f7-a952-7cefd07de80f:/dev/nvme0n1p4 6944c52c-7d25-465c-9e6f-18810cdb2523:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 13 20:53:59.399046 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.398964    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 20:53:59.403445 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.403410    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 20:53:59.405549 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.405442    2556 manager.go:216] Machine: {Timestamp:2026-07-13 20:53:59.40424178 +0000 UTC m=+0.342446079 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101776 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec225217035e039b3a4923f68c4c2897 SystemUUID:ec225217-035e-039b-3a49-23f68c4c2897 BootID:12dc4b2d-6675-4e1f-becb-4a43e0849af2 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:33:b4:3c:be:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:33:b4:3c:be:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:7c:80:79:dc:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 20:53:59.405549 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.405538    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 20:53:59.405665 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.405610    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 20:53:59.407983 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.407962    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 20:53:59.408105 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.407986    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-146.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 20:53:59.408692 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.408682    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 20:53:59.408721 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.408693    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 20:53:59.408721 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.408700    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 20:53:59.408721 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.408713    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 20:53:59.409930 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.409921    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 20:53:59.412953 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.412943    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 20:53:59.413060 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413052    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 20:53:59.413092 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413079    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 20:53:59.413092 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413090    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 20:53:59.413138 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413108    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 20:53:59.413138 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413117    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 13 20:53:59.413138 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413131    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 20:53:59.413545 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413532    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 20:53:59.413589 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.413549    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 20:53:59.416029 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.416011    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 20:53:59.417629 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417616    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 20:53:59.417668 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417645    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 20:53:59.417789 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417780    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 20:53:59.417819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417793    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 20:53:59.417819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417799    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 20:53:59.417819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417804    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 20:53:59.417819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417811    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 20:53:59.417819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.417816    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 20:53:59.418296 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.418287    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 20:53:59.418354 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.418299    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 20:53:59.418354 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.418307    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 20:53:59.418354 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.418312    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 20:53:59.418354 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.418320    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 20:53:59.418354 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.418327    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 20:53:59.419470 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.419460    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 20:53:59.419470 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.419471    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 20:53:59.422167 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.422153    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-146.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 20:53:59.422225 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.422175    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 20:53:59.422538 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.422512    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 20:53:59.422646 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.422635    2556 server.go:1268] "Started kubelet"
Jul 13 20:53:59.422826 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.422788    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 20:53:59.422898 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.422815    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 20:53:59.422977 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.422904    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mr6s9"
Jul 13 20:53:59.422977 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.422920    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 20:53:59.423424 ip-10-0-138-146 systemd[1]: Started Kubernetes Kubelet.
Jul 13 20:53:59.423929 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.423848    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 20:53:59.425133 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.425121    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 20:53:59.426974 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.426959    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 20:53:59.428544 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.428529    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 20:53:59.428622 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.428532    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 20:53:59.428681 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.428661    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:53:59.429139 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429121    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 20:53:59.429139 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429121    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 20:53:59.429267 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429150    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 20:53:59.429267 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429234    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 20:53:59.429267 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429243    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 20:53:59.429420 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429368    2556 factory.go:55] Registering systemd factory
Jul 13 20:53:59.429420 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429381    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 13 20:53:59.429594 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429579    2556 factory.go:153] Registering CRI-O factory
Jul 13 20:53:59.429594 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429593    2556 factory.go:223] Registration of the crio container factory successfully
Jul 13 20:53:59.429708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429629    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 20:53:59.429708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429649    2556 factory.go:103] Registering Raw factory
Jul 13 20:53:59.429708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429665    2556 manager.go:1196] Started watching for new ooms in manager
Jul 13 20:53:59.429840 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.429785    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mr6s9"
Jul 13 20:53:59.430243 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.430228    2556 manager.go:349] Starting recovery of all containers
Jul 13 20:53:59.430825 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.429837    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-146.ec2.internal.18c1f4c18597da98  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-146.ec2.internal,UID:ip-10-0-138-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-146.ec2.internal,},FirstTimestamp:2026-07-13 20:53:59.422614168 +0000 UTC m=+0.360818523,LastTimestamp:2026-07-13 20:53:59.422614168 +0000 UTC m=+0.360818523,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-146.ec2.internal,}"
Jul 13 20:53:59.434202 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.434174    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 20:53:59.434202 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.434179    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 20:53:59.436617 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.436593    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 20:53:59.441412 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.441386    2556 manager.go:354] Recovery completed
Jul 13 20:53:59.446510 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.446422    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 20:53:59.448354 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.448323    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 20:53:59.448416 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.448364    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 20:53:59.448416 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.448375    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientPID"
Jul 13 20:53:59.448839 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.448827    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 20:53:59.448839 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.448838    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 20:53:59.448937 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.448856    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 20:53:59.448937 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.448839    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 20:53:59.449531 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.449517    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 20:53:59.449588 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.449548    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 20:53:59.449588 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.449557    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientPID"
Jul 13 20:53:59.449924 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.449913    2556 policy_none.go:49] "None policy: Start"
Jul 13 20:53:59.449958 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.449927    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 20:53:59.449958 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.449936    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 20:53:59.450754 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.450744    2556 policy_none.go:47] "Start"
Jul 13 20:53:59.486088 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.486073    2556 manager.go:337] "Starting Device Plugin manager"
Jul 13 20:53:59.486170 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.486114    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 20:53:59.486170 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.486126    2556 server.go:87] "Starting device plugin registration server"
Jul 13 20:53:59.486391 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.486381    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 20:53:59.486422 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.486394    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 20:53:59.486471 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.486459    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 20:53:59.486588 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.486528    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 20:53:59.486588 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.486539    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 20:53:59.486988 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.486971    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 20:53:59.487059 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.487013    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:53:59.586686 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.586647    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 20:53:59.588267 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.588244    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 20:53:59.588370 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.588278    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 20:53:59.588370 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.588287    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientPID"
Jul 13 20:53:59.588370 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.588313    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.593369 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.593353    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.593448 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.593378    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-146.ec2.internal\": node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:53:59.596254 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.596237    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 20:53:59.596331 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.596271    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 20:53:59.596331 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.596287    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 20:53:59.596331 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.596321    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 20:53:59.598169 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.598155    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 20:53:59.610745 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.610732    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:53:59.697061 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.697015    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"]
Jul 13 20:53:59.697237 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.697106    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 20:53:59.698861 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.698844    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 20:53:59.698946 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.698870    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 20:53:59.698946 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.698879    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientPID"
Jul 13 20:53:59.700128 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.700117    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 20:53:59.700233 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.700217    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.700276 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.700248    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 20:53:59.701003 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.700974    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 20:53:59.701003 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.701001    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 20:53:59.701120 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.700981    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 20:53:59.701120 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.701013    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientPID"
Jul 13 20:53:59.701120 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.701031    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 20:53:59.701120 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.701042    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientPID"
Jul 13 20:53:59.702424 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.702409    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.702513 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.702432    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 20:53:59.703013 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.702998    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 20:53:59.703095 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.703023    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 20:53:59.703095 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.703035    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeHasSufficientPID"
Jul 13 20:53:59.711059 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.711045    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:53:59.723815 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.723798    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-146.ec2.internal\" not found" node="ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.727631 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.727618    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-146.ec2.internal\" not found" node="ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.730763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.730749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eac84a200e5292e5d94479fe1961055a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal\" (UID: \"eac84a200e5292e5d94479fe1961055a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.730816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.730771    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eac84a200e5292e5d94479fe1961055a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal\" (UID: \"eac84a200e5292e5d94479fe1961055a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.730816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.730787    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d10ad1489d9712613b909680a7e53bfb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-146.ec2.internal\" (UID: \"d10ad1489d9712613b909680a7e53bfb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.811518 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.811494    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:53:59.831852 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.831828    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d10ad1489d9712613b909680a7e53bfb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-146.ec2.internal\" (UID: \"d10ad1489d9712613b909680a7e53bfb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.831926 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.831876    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eac84a200e5292e5d94479fe1961055a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal\" (UID: \"eac84a200e5292e5d94479fe1961055a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.831926 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.831898    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eac84a200e5292e5d94479fe1961055a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal\" (UID: \"eac84a200e5292e5d94479fe1961055a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.831995 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.831927    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d10ad1489d9712613b909680a7e53bfb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-146.ec2.internal\" (UID: \"d10ad1489d9712613b909680a7e53bfb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.831995 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.831936    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eac84a200e5292e5d94479fe1961055a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal\" (UID: \"eac84a200e5292e5d94479fe1961055a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.831995 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:53:59.831989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eac84a200e5292e5d94479fe1961055a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal\" (UID: \"eac84a200e5292e5d94479fe1961055a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:53:59.912062 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:53:59.911995    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.012517 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.012484    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.025647 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.025630    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal"
Jul 13 20:54:00.029693 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.029668    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:54:00.113023 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.112989    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.213512 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.213443    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.284595 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.284573    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 20:54:00.314204 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.314179    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.374534 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.374503    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 20:54:00.374973 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.374662    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 20:54:00.374973 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.374685    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 20:54:00.414762 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.414734    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.433312 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.433276    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 20:48:59 +0000 UTC" deadline="2028-02-02 18:32:13.980861464 +0000 UTC"
Jul 13 20:54:00.433312 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.433309    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13653h38m13.54755556s"
Jul 13 20:54:00.501266 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:00.501233    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10ad1489d9712613b909680a7e53bfb.slice/crio-6bc8300db3a8b3a94fe76326af6f4c2babf96de71cc8ec99f419c67e468cc680 WatchSource:0}: Error finding container 6bc8300db3a8b3a94fe76326af6f4c2babf96de71cc8ec99f419c67e468cc680: Status 404 returned error can't find the container with id 6bc8300db3a8b3a94fe76326af6f4c2babf96de71cc8ec99f419c67e468cc680
Jul 13 20:54:00.501592 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:00.501566    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac84a200e5292e5d94479fe1961055a.slice/crio-6154c3373ae1ffe3cd9f322617b313bfef5293b56bcaa84e526ee6d013360a38 WatchSource:0}: Error finding container 6154c3373ae1ffe3cd9f322617b313bfef5293b56bcaa84e526ee6d013360a38: Status 404 returned error can't find the container with id 6154c3373ae1ffe3cd9f322617b313bfef5293b56bcaa84e526ee6d013360a38
Jul 13 20:54:00.506447 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.506433    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 20:54:00.515186 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.515169    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.599709 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.599663    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal" event={"ID":"eac84a200e5292e5d94479fe1961055a","Type":"ContainerStarted","Data":"6154c3373ae1ffe3cd9f322617b313bfef5293b56bcaa84e526ee6d013360a38"}
Jul 13 20:54:00.599950 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.599933    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 20:54:00.600561 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.600543    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal" event={"ID":"d10ad1489d9712613b909680a7e53bfb","Type":"ContainerStarted","Data":"6bc8300db3a8b3a94fe76326af6f4c2babf96de71cc8ec99f419c67e468cc680"}
Jul 13 20:54:00.615745 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.615722    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.716226 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.716188    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.816653 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:00.816579    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-146.ec2.internal\" not found"
Jul 13 20:54:00.883741 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.883712    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 20:54:00.929902 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.929879    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal"
Jul 13 20:54:00.939834 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.939809    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 20:54:00.941092 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.941076    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal"
Jul 13 20:54:00.951139 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:00.951122    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 20:54:01.414437 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.414370    2556 apiserver.go:52] "Watching apiserver"
Jul 13 20:54:01.422158 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.421978    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 20:54:01.422481 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.422455    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-hbzmm","openshift-dns/node-resolver-n688c","openshift-multus/multus-additional-cni-plugins-zmwk4","openshift-multus/multus-lv4m5","openshift-multus/network-metrics-daemon-grcrp","openshift-network-operator/iptables-alerter-llzq6","kube-system/global-pull-secret-syncer-2z72g","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj","openshift-image-registry/node-ca-9pjnw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal","openshift-network-diagnostics/network-check-target-j28gz","openshift-ovn-kubernetes/ovnkube-node-qwm4s","kube-system/konnectivity-agent-9kv8m","kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal"]
Jul 13 20:54:01.423881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.423859    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.423993 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.423946    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:01.424945 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.424928    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.426200 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.426180    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.427552 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.427531    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-slbvw\""
Jul 13 20:54:01.427783 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.427767    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 20:54:01.427988 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.427973    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 20:54:01.428101 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.428085    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.428706 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.428595    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 20:54:01.428792 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.428600    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:01.428792 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.428739    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 20:54:01.428792 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.428782    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:01.431015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.429413    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 20:54:01.431015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.429470    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 20:54:01.431015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.429598    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 20:54:01.431015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.430731    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l6mpf\""
Jul 13 20:54:01.431015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.430989    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 20:54:01.431294 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.431136    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tkc2v\""
Jul 13 20:54:01.431294 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.431273    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 20:54:01.432323 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.431653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.432986 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.432926    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.434292 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.434268    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dv6bh\""
Jul 13 20:54:01.434396 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.434317    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 20:54:01.434578 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.434558    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 20:54:01.434680 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.434581    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 20:54:01.435215 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.434801    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.435544 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.435525    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lp54v\""
Jul 13 20:54:01.435746 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.435728    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 20:54:01.435860 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.435844    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 20:54:01.435918 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.435844    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 20:54:01.436024 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.436008    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.437821 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.437515    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c9zsw\""
Jul 13 20:54:01.437821 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.437551    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 20:54:01.437821 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.437798    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.438075 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.438059    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 20:54:01.438579 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.438437    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 20:54:01.438579 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.438506    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z9jmf\""
Jul 13 20:54:01.438860 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.438845    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 20:54:01.439092 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439076    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:01.439167 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.439148    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:01.439167 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439074    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-kubelet\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439306 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439193    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-daemon-config\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439306 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439220    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-etc-kubernetes\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439306 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhc8x\" (UniqueName: \"kubernetes.io/projected/de4035ca-3984-435e-8387-217677bcc954-kube-api-access-rhc8x\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:01.439306 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439272    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.439306 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439297    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439321    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cnibin\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439357    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-cni-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439381    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-cnibin\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439403    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-os-release\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439426    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-socket-dir-parent\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439447    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-conf-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439471    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439496    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-cni-bin\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439536    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-multus-certs\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.439568 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439572    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a0d98d7-d4e9-4395-9ead-400cc91688ac-host-slash\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439588    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxbmv\" (UniqueName: \"kubernetes.io/projected/0a0d98d7-d4e9-4395-9ead-400cc91688ac-kube-api-access-cxbmv\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439603    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-k8s-cni-cncf-io\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439627    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439644    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrqmg\" (UniqueName: \"kubernetes.io/projected/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-kube-api-access-zrqmg\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439659    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-hostroot\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439701    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ndrw\" (UniqueName: \"kubernetes.io/projected/21e605a3-44ba-43d8-b5e5-4e365565ed28-kube-api-access-7ndrw\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439756    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0420276d-4854-4250-9a5d-5ebdfad81a34-kubelet-config\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439806    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0420276d-4854-4250-9a5d-5ebdfad81a34-dbus\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439838    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f457d97c-0c11-4379-ac4a-6d8033a4cff7-agent-certs\") pod \"konnectivity-agent-9kv8m\" (UID: \"f457d97c-0c11-4379-ac4a-6d8033a4cff7\") " pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439862    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-os-release\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439889    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-system-cni-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439910    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21e605a3-44ba-43d8-b5e5-4e365565ed28-cni-binary-copy\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439932    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-netns\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439953    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-cni-multus\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.440004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.439982    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:01.440712 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.440011    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a0d98d7-d4e9-4395-9ead-400cc91688ac-iptables-alerter-script\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.440712 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.440058    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f457d97c-0c11-4379-ac4a-6d8033a4cff7-konnectivity-ca\") pod \"konnectivity-agent-9kv8m\" (UID: \"f457d97c-0c11-4379-ac4a-6d8033a4cff7\") " pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.440712 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.440087    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.440712 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.440116    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.440712 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.440374    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.440712 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.440635    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 20:54:01.440712 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.440708    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 20:54:01.441041 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.441020    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 20:54:01.441233 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.441216    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g5b22\""
Jul 13 20:54:01.443713 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.443692    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 20:54:01.445051 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.444990    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 20:54:01.445539 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.445358    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 20:54:01.445539 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.445371    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 20:54:01.445539 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.445412    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n8rlf\""
Jul 13 20:54:01.445539 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.445358    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 20:54:01.445774 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.445763    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 20:54:01.447815 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.447767    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 20:54:01.462426 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.462404    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pn247"
Jul 13 20:54:01.469938 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.469900    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pn247"
Jul 13 20:54:01.531055 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.531029    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 20:54:01.540648 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540514    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0420276d-4854-4250-9a5d-5ebdfad81a34-kubelet-config\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.540648 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540571    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0420276d-4854-4250-9a5d-5ebdfad81a34-dbus\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.540648 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540601    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-netns\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.540648 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540629    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-cni-multus\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-sys\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540690    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540717    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-daemon-config\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540743    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-kubelet\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540766    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-log-socket\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540795    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/38b9c8cd-9d66-430c-b411-e92d0a730deb-hosts-file\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540820    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540844    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-run-netns\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540870    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-var-lib-kubelet\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540897    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-cni-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-cnibin\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540949    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-socket-dir-parent\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.540979 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.540974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-conf-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541001    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-kubernetes\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541030    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541065    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-cni-bin\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541101    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-systemd-units\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541128    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-slash\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541153    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-ovn\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541178    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-ovnkube-script-lib\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541208    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svdkj\" (UniqueName: \"kubernetes.io/projected/9da8359f-85cf-45d3-a81e-82a48d894872-kube-api-access-svdkj\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541234    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-device-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541260    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541285    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-serviceca\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541313    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-modprobe-d\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541355    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8klz\" (UniqueName: \"kubernetes.io/projected/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-kube-api-access-j8klz\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541416    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrqmg\" (UniqueName: \"kubernetes.io/projected/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-kube-api-access-zrqmg\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.541529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541444    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-hostroot\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541470    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ndrw\" (UniqueName: \"kubernetes.io/projected/21e605a3-44ba-43d8-b5e5-4e365565ed28-kube-api-access-7ndrw\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541500    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-cni-netd\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541525    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-host\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-etc-kubernetes\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541578    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541617    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f457d97c-0c11-4379-ac4a-6d8033a4cff7-agent-certs\") pod \"konnectivity-agent-9kv8m\" (UID: \"f457d97c-0c11-4379-ac4a-6d8033a4cff7\") " pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541644    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-os-release\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-system-cni-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541696    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21e605a3-44ba-43d8-b5e5-4e365565ed28-cni-binary-copy\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541720    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-cni-bin\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz8rk\" (UniqueName: \"kubernetes.io/projected/38b9c8cd-9d66-430c-b411-e92d0a730deb-kube-api-access-sz8rk\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541786    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541836    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a0d98d7-d4e9-4395-9ead-400cc91688ac-iptables-alerter-script\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541932    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-hostroot\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541942    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-netns\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541949    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0420276d-4854-4250-9a5d-5ebdfad81a34-kubelet-config\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.542015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.541985    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-cnibin\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542001    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-system-cni-dir\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542037    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-conf-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.542074    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542089    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-cni-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542140    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-socket-dir-parent\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.542176    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:02.042137699 +0000 UTC m=+2.980342009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542189    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-cni-multus\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542073    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0420276d-4854-4250-9a5d-5ebdfad81a34-dbus\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542325    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-cni-bin\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542397    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-system-cni-dir\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f457d97c-0c11-4379-ac4a-6d8033a4cff7-konnectivity-ca\") pod \"konnectivity-agent-9kv8m\" (UID: \"f457d97c-0c11-4379-ac4a-6d8033a4cff7\") " pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542444    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542463    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/21e605a3-44ba-43d8-b5e5-4e365565ed28-multus-daemon-config\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-kubelet\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542508    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9da8359f-85cf-45d3-a81e-82a48d894872-ovn-node-metrics-cert\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542539    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhc8x\" (UniqueName: \"kubernetes.io/projected/de4035ca-3984-435e-8387-217677bcc954-kube-api-access-rhc8x\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:01.542619 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542560    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-os-release\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542696    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542734    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-var-lib-kubelet\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542786    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-etc-kubernetes\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542888    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542922    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.542994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543039    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-var-lib-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543067    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-etc-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543100    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qts\" (UniqueName: \"kubernetes.io/projected/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-kube-api-access-j6qts\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543123    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-systemd\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a0d98d7-d4e9-4395-9ead-400cc91688ac-iptables-alerter-script\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543148    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cnibin\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543287    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-os-release\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.543291 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.543287    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543322    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cnibin\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.543378    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret podName:0420276d-4854-4250-9a5d-5ebdfad81a34 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:02.04333015 +0000 UTC m=+2.981534455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret") pod "global-pull-secret-syncer-2z72g" (UID: "0420276d-4854-4250-9a5d-5ebdfad81a34") : object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543412    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-node-log\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543423    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21e605a3-44ba-43d8-b5e5-4e365565ed28-cni-binary-copy\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543479    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-os-release\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543543    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-ovnkube-config\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543586    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysctl-d\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543621    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysctl-conf\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543649    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-tuned\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543709    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-multus-certs\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543743    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-registration-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543798    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-multus-certs\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543812    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-host\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543846    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysconfig\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.543993 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543865    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-tmp\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543896    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a0d98d7-d4e9-4395-9ead-400cc91688ac-host-slash\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543940    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a0d98d7-d4e9-4395-9ead-400cc91688ac-host-slash\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.543970    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxbmv\" (UniqueName: \"kubernetes.io/projected/0a0d98d7-d4e9-4395-9ead-400cc91688ac-kube-api-access-cxbmv\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-k8s-cni-cncf-io\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544018    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8tvw\" (UniqueName: \"kubernetes.io/projected/1efd37ff-1446-46de-8310-3e3589c2246b-kube-api-access-j8tvw\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544043    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-systemd\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544044    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f457d97c-0c11-4379-ac4a-6d8033a4cff7-konnectivity-ca\") pod \"konnectivity-agent-9kv8m\" (UID: \"f457d97c-0c11-4379-ac4a-6d8033a4cff7\") " pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544067    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-run-ovn-kubernetes\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544083    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/21e605a3-44ba-43d8-b5e5-4e365565ed28-host-run-k8s-cni-cncf-io\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544091    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b9c8cd-9d66-430c-b411-e92d0a730deb-tmp-dir\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544113    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544137    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-socket-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544160    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-sys-fs\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544180    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-run\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544207    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.544600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544231    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-env-overrides\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.545148 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544251    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-lib-modules\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.545148 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.544589    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.548026 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.548000    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f457d97c-0c11-4379-ac4a-6d8033a4cff7-agent-certs\") pod \"konnectivity-agent-9kv8m\" (UID: \"f457d97c-0c11-4379-ac4a-6d8033a4cff7\") " pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.550600 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.550578    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrqmg\" (UniqueName: \"kubernetes.io/projected/fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2-kube-api-access-zrqmg\") pod \"multus-additional-cni-plugins-zmwk4\" (UID: \"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2\") " pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.551821 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.551801    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ndrw\" (UniqueName: \"kubernetes.io/projected/21e605a3-44ba-43d8-b5e5-4e365565ed28-kube-api-access-7ndrw\") pod \"multus-lv4m5\" (UID: \"21e605a3-44ba-43d8-b5e5-4e365565ed28\") " pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.554079 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.554055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhc8x\" (UniqueName: \"kubernetes.io/projected/de4035ca-3984-435e-8387-217677bcc954-kube-api-access-rhc8x\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:01.554911 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.554890    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxbmv\" (UniqueName: \"kubernetes.io/projected/0a0d98d7-d4e9-4395-9ead-400cc91688ac-kube-api-access-cxbmv\") pod \"iptables-alerter-llzq6\" (UID: \"0a0d98d7-d4e9-4395-9ead-400cc91688ac\") " pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.603974 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.603946    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 20:54:01.644784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644745    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-kubernetes\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.644784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644789    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-systemd-units\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644840    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-systemd-units\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644868    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-kubernetes\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644891    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-slash\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644927    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-slash\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644931    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-ovn\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644952    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-ovnkube-script-lib\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644967    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svdkj\" (UniqueName: \"kubernetes.io/projected/9da8359f-85cf-45d3-a81e-82a48d894872-kube-api-access-svdkj\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.644994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.644982    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-device-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645035    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-device-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645064    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645093    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-serviceca\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-ovn\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645130    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-modprobe-d\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645152    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645157    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8klz\" (UniqueName: \"kubernetes.io/projected/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-kube-api-access-j8klz\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645194    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645222    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-cni-netd\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645246    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-host\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645283    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645314    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-cni-bin\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645361 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz8rk\" (UniqueName: \"kubernetes.io/projected/38b9c8cd-9d66-430c-b411-e92d0a730deb-kube-api-access-sz8rk\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645404    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9da8359f-85cf-45d3-a81e-82a48d894872-ovn-node-metrics-cert\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645437    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645452    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-var-lib-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645357    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-host\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645476    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-cni-netd\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645478    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-etc-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645476    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-modprobe-d\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645504    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qts\" (UniqueName: \"kubernetes.io/projected/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-kube-api-access-j6qts\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645530    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-systemd\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645530    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-var-lib-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645565    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-etc-openvswitch\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645604    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-node-log\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645616    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-ovnkube-script-lib\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645630    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-ovnkube-config\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645650    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-cni-bin\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645664    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysctl-d\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-node-log\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.645816 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645690    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysctl-conf\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-systemd\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-serviceca\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645716    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-tuned\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645762    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-registration-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645786    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysctl-d\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645825    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-host\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645826    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysctl-conf\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645788    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-host\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645880    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysconfig\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645906    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-tmp\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645835    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-registration-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-sysconfig\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646088    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-ovnkube-config\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.645955    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8tvw\" (UniqueName: \"kubernetes.io/projected/1efd37ff-1446-46de-8310-3e3589c2246b-kube-api-access-j8tvw\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-systemd\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646525    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-run-ovn-kubernetes\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b9c8cd-9d66-430c-b411-e92d0a730deb-tmp-dir\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.646708 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646573    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-socket-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646597    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-sys-fs\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-run\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646680    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-env-overrides\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646703    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-lib-modules\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646729    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-sys\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646753    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-kubelet\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646774    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-log-socket\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646797    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/38b9c8cd-9d66-430c-b411-e92d0a730deb-hosts-file\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646818    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646840    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-run-netns\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646864    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-var-lib-kubelet\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646947    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-var-lib-kubelet\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.646993    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-run-systemd\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.647031    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-run-ovn-kubernetes\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.647279    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b9c8cd-9d66-430c-b411-e92d0a730deb-tmp-dir\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.647562 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.647410    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-socket-dir\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.647459    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-sys-fs\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.647502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-run\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.647540    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648038    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-tmp\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648154    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-log-socket\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648212    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-etc-tuned\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648253    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9da8359f-85cf-45d3-a81e-82a48d894872-ovn-node-metrics-cert\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648330    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1efd37ff-1446-46de-8310-3e3589c2246b-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.648387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648365    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-run-netns\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.648807 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648419    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/38b9c8cd-9d66-430c-b411-e92d0a730deb-hosts-file\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.648807 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648433    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-sys\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.648807 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648475    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9da8359f-85cf-45d3-a81e-82a48d894872-host-kubelet\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.648807 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648561    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-lib-modules\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.648807 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.648567    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9da8359f-85cf-45d3-a81e-82a48d894872-env-overrides\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.651586 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.651556    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 20:54:01.651586 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.651583    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 20:54:01.651785 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.651598    2556 projected.go:196] Error preparing data for projected volume kube-api-access-vfs46 for pod openshift-network-diagnostics/network-check-target-j28gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:01.651785 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:01.651710    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46 podName:d6c59737-a9b2-45fa-8bf6-4691e0d37c77 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:02.151690302 +0000 UTC m=+3.089894608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vfs46" (UniqueName: "kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46") pod "network-check-target-j28gz" (UID: "d6c59737-a9b2-45fa-8bf6-4691e0d37c77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:01.653839 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.653814    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz8rk\" (UniqueName: \"kubernetes.io/projected/38b9c8cd-9d66-430c-b411-e92d0a730deb-kube-api-access-sz8rk\") pod \"node-resolver-n688c\" (UID: \"38b9c8cd-9d66-430c-b411-e92d0a730deb\") " pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.654037 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.654005    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8klz\" (UniqueName: \"kubernetes.io/projected/6f25a02d-8cc0-406c-a5c1-670ee5d395fb-kube-api-access-j8klz\") pod \"tuned-hbzmm\" (UID: \"6f25a02d-8cc0-406c-a5c1-670ee5d395fb\") " pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.654628 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.654596    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8tvw\" (UniqueName: \"kubernetes.io/projected/1efd37ff-1446-46de-8310-3e3589c2246b-kube-api-access-j8tvw\") pod \"aws-ebs-csi-driver-node-8hncj\" (UID: \"1efd37ff-1446-46de-8310-3e3589c2246b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.654740 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.654696    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qts\" (UniqueName: \"kubernetes.io/projected/3ba71b3d-1630-4cd4-912c-2ed8f84aad46-kube-api-access-j6qts\") pod \"node-ca-9pjnw\" (UID: \"3ba71b3d-1630-4cd4-912c-2ed8f84aad46\") " pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.655109 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.655091    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svdkj\" (UniqueName: \"kubernetes.io/projected/9da8359f-85cf-45d3-a81e-82a48d894872-kube-api-access-svdkj\") pod \"ovnkube-node-qwm4s\" (UID: \"9da8359f-85cf-45d3-a81e-82a48d894872\") " pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:01.741633 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.741542    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:01.748325 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.748302    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zmwk4"
Jul 13 20:54:01.757885 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.757864    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lv4m5"
Jul 13 20:54:01.764564 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.764525    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llzq6"
Jul 13 20:54:01.772179 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.772160    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9pjnw"
Jul 13 20:54:01.779920 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.779901    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbzmm"
Jul 13 20:54:01.787493 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.787474    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n688c"
Jul 13 20:54:01.796218 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.796202    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj"
Jul 13 20:54:01.804793 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:01.804773    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:02.049580 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.049507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:02.049580 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.049544    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:02.049781 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.049660    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:02.049781 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.049721    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:03.049706284 +0000 UTC m=+3.987910577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:02.049781 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.049660    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:02.049894 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.049834    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret podName:0420276d-4854-4250-9a5d-5ebdfad81a34 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:03.049812245 +0000 UTC m=+3.988016548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret") pod "global-pull-secret-syncer-2z72g" (UID: "0420276d-4854-4250-9a5d-5ebdfad81a34") : object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:02.187876 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.187842    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 20:54:02.251436 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.251399    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:02.251607 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.251592    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 20:54:02.251659 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.251621    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 20:54:02.251659 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.251634    2556 projected.go:196] Error preparing data for projected volume kube-api-access-vfs46 for pod openshift-network-diagnostics/network-check-target-j28gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:02.251729 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.251701    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46 podName:d6c59737-a9b2-45fa-8bf6-4691e0d37c77 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:03.251680907 +0000 UTC m=+4.189885197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfs46" (UniqueName: "kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46") pod "network-check-target-j28gz" (UID: "d6c59737-a9b2-45fa-8bf6-4691e0d37c77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:02.472716 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.472649    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:49:01 +0000 UTC" deadline="2027-12-20 10:07:20.865246656 +0000 UTC"
Jul 13 20:54:02.472716 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.472709    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12589h13m18.392540709s"
Jul 13 20:54:02.523232 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.523194    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf457d97c_0c11_4379_ac4a_6d8033a4cff7.slice/crio-8501d67291bfb5dd2a482109d21b80aad4573ae0155c142ee2abc5fac7d7bec6 WatchSource:0}: Error finding container 8501d67291bfb5dd2a482109d21b80aad4573ae0155c142ee2abc5fac7d7bec6: Status 404 returned error can't find the container with id 8501d67291bfb5dd2a482109d21b80aad4573ae0155c142ee2abc5fac7d7bec6
Jul 13 20:54:02.524412 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.524354    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0d98d7_d4e9_4395_9ead_400cc91688ac.slice/crio-2c121871ed61899d000f34a3d950c4b11da98580c88c446b72198a75c3c3b374 WatchSource:0}: Error finding container 2c121871ed61899d000f34a3d950c4b11da98580c88c446b72198a75c3c3b374: Status 404 returned error can't find the container with id 2c121871ed61899d000f34a3d950c4b11da98580c88c446b72198a75c3c3b374
Jul 13 20:54:02.525631 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.525604    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8cfbe1_0c58_4bb5_94b9_1d536f62aaf2.slice/crio-06ec6d96ff910b04f702075e762c3d55b333205672a9213bb96ddbe7f906d754 WatchSource:0}: Error finding container 06ec6d96ff910b04f702075e762c3d55b333205672a9213bb96ddbe7f906d754: Status 404 returned error can't find the container with id 06ec6d96ff910b04f702075e762c3d55b333205672a9213bb96ddbe7f906d754
Jul 13 20:54:02.528997 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.528970    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b9c8cd_9d66_430c_b411_e92d0a730deb.slice/crio-6e9721f4bf21a9059611c37d93b8a770679cc4e5deebc0768f2dd4054f9b941c WatchSource:0}: Error finding container 6e9721f4bf21a9059611c37d93b8a770679cc4e5deebc0768f2dd4054f9b941c: Status 404 returned error can't find the container with id 6e9721f4bf21a9059611c37d93b8a770679cc4e5deebc0768f2dd4054f9b941c
Jul 13 20:54:02.530252 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.530231    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e605a3_44ba_43d8_b5e5_4e365565ed28.slice/crio-950d86a3292a0abcc050404abc4d4a868d0bc8fc71b6774d283b753d1e2bf9ee WatchSource:0}: Error finding container 950d86a3292a0abcc050404abc4d4a868d0bc8fc71b6774d283b753d1e2bf9ee: Status 404 returned error can't find the container with id 950d86a3292a0abcc050404abc4d4a868d0bc8fc71b6774d283b753d1e2bf9ee
Jul 13 20:54:02.531927 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.531827    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1efd37ff_1446_46de_8310_3e3589c2246b.slice/crio-474181fc84ff31b6bdd52bef9c237a34086b8f4c50b3cb63ca1a7c956e103163 WatchSource:0}: Error finding container 474181fc84ff31b6bdd52bef9c237a34086b8f4c50b3cb63ca1a7c956e103163: Status 404 returned error can't find the container with id 474181fc84ff31b6bdd52bef9c237a34086b8f4c50b3cb63ca1a7c956e103163
Jul 13 20:54:02.533254 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.533232    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba71b3d_1630_4cd4_912c_2ed8f84aad46.slice/crio-7afe46986e65f4cc94521945b0a7a98c64aeb5aa950b616cc0d9b32b1f356c68 WatchSource:0}: Error finding container 7afe46986e65f4cc94521945b0a7a98c64aeb5aa950b616cc0d9b32b1f356c68: Status 404 returned error can't find the container with id 7afe46986e65f4cc94521945b0a7a98c64aeb5aa950b616cc0d9b32b1f356c68
Jul 13 20:54:02.555638 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.555613    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f25a02d_8cc0_406c_a5c1_670ee5d395fb.slice/crio-05d48943e1eede4291008685ffac133584456e4363c3c884cd808b6a14a678f4 WatchSource:0}: Error finding container 05d48943e1eede4291008685ffac133584456e4363c3c884cd808b6a14a678f4: Status 404 returned error can't find the container with id 05d48943e1eede4291008685ffac133584456e4363c3c884cd808b6a14a678f4
Jul 13 20:54:02.556372 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:02.556330    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da8359f_85cf_45d3_a81e_82a48d894872.slice/crio-671e4b04d2d9f89e820a8877c0b07bf807471de2d8009f18141c72380a954f15 WatchSource:0}: Error finding container 671e4b04d2d9f89e820a8877c0b07bf807471de2d8009f18141c72380a954f15: Status 404 returned error can't find the container with id 671e4b04d2d9f89e820a8877c0b07bf807471de2d8009f18141c72380a954f15
Jul 13 20:54:02.597248 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.597129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:02.597393 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:02.597334    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:02.604282 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.604254    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lv4m5" event={"ID":"21e605a3-44ba-43d8-b5e5-4e365565ed28","Type":"ContainerStarted","Data":"950d86a3292a0abcc050404abc4d4a868d0bc8fc71b6774d283b753d1e2bf9ee"}
Jul 13 20:54:02.605330 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.605307    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj" event={"ID":"1efd37ff-1446-46de-8310-3e3589c2246b","Type":"ContainerStarted","Data":"474181fc84ff31b6bdd52bef9c237a34086b8f4c50b3cb63ca1a7c956e103163"}
Jul 13 20:54:02.606354 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.606322    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n688c" event={"ID":"38b9c8cd-9d66-430c-b411-e92d0a730deb","Type":"ContainerStarted","Data":"6e9721f4bf21a9059611c37d93b8a770679cc4e5deebc0768f2dd4054f9b941c"}
Jul 13 20:54:02.607494 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.607470    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerStarted","Data":"06ec6d96ff910b04f702075e762c3d55b333205672a9213bb96ddbe7f906d754"}
Jul 13 20:54:02.608631 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.608610    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9kv8m" event={"ID":"f457d97c-0c11-4379-ac4a-6d8033a4cff7","Type":"ContainerStarted","Data":"8501d67291bfb5dd2a482109d21b80aad4573ae0155c142ee2abc5fac7d7bec6"}
Jul 13 20:54:02.609873 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.609852    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbzmm" event={"ID":"6f25a02d-8cc0-406c-a5c1-670ee5d395fb","Type":"ContainerStarted","Data":"05d48943e1eede4291008685ffac133584456e4363c3c884cd808b6a14a678f4"}
Jul 13 20:54:02.612146 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.612122    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9pjnw" event={"ID":"3ba71b3d-1630-4cd4-912c-2ed8f84aad46","Type":"ContainerStarted","Data":"7afe46986e65f4cc94521945b0a7a98c64aeb5aa950b616cc0d9b32b1f356c68"}
Jul 13 20:54:02.613805 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.613784    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"671e4b04d2d9f89e820a8877c0b07bf807471de2d8009f18141c72380a954f15"}
Jul 13 20:54:02.615629 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:02.615608    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llzq6" event={"ID":"0a0d98d7-d4e9-4395-9ead-400cc91688ac","Type":"ContainerStarted","Data":"2c121871ed61899d000f34a3d950c4b11da98580c88c446b72198a75c3c3b374"}
Jul 13 20:54:03.056275 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.056191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:03.056275 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.056244    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:03.056550 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.056389    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:03.056550 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.056447    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret podName:0420276d-4854-4250-9a5d-5ebdfad81a34 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:05.056429206 +0000 UTC m=+5.994633496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret") pod "global-pull-secret-syncer-2z72g" (UID: "0420276d-4854-4250-9a5d-5ebdfad81a34") : object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:03.056850 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.056830    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:03.056937 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.056889    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:05.056871887 +0000 UTC m=+5.995076190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:03.257920 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.257882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:03.258129 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.258100    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 20:54:03.258129 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.258126    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 20:54:03.258260 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.258141    2556 projected.go:196] Error preparing data for projected volume kube-api-access-vfs46 for pod openshift-network-diagnostics/network-check-target-j28gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:03.258260 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.258201    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46 podName:d6c59737-a9b2-45fa-8bf6-4691e0d37c77 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:05.258184123 +0000 UTC m=+6.196388432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfs46" (UniqueName: "kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46") pod "network-check-target-j28gz" (UID: "d6c59737-a9b2-45fa-8bf6-4691e0d37c77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:03.473449 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.473412    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:49:01 +0000 UTC" deadline="2028-01-07 15:55:35.050577369 +0000 UTC"
Jul 13 20:54:03.473449 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.473448    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13027h1m31.577133719s"
Jul 13 20:54:03.597483 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.597456    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:03.597625 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.597585    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:03.598063 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.598043    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:03.598164 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:03.598144    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:03.623853 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.623810    2556 generic.go:356] "Generic (PLEG): container finished" podID="eac84a200e5292e5d94479fe1961055a" containerID="62bb15f3c126b60ab3e1b0b1eebd57ac621854058fcb5c3013303158899bc0bc" exitCode=0
Jul 13 20:54:03.624095 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.624067    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal" event={"ID":"eac84a200e5292e5d94479fe1961055a","Type":"ContainerDied","Data":"62bb15f3c126b60ab3e1b0b1eebd57ac621854058fcb5c3013303158899bc0bc"}
Jul 13 20:54:03.638561 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.638075    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal" event={"ID":"d10ad1489d9712613b909680a7e53bfb","Type":"ContainerStarted","Data":"d0ed22078642ea55380a2d708df03d33417d606cb1c0af24b23e0f06907af991"}
Jul 13 20:54:03.659037 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:03.658805    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-146.ec2.internal" podStartSLOduration=3.658788601 podStartE2EDuration="3.658788601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:03.658612432 +0000 UTC m=+4.596816743" watchObservedRunningTime="2026-07-13 20:54:03.658788601 +0000 UTC m=+4.596992913"
Jul 13 20:54:04.597878 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:04.597386    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:04.597878 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:04.597526    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:04.662478 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:04.662418    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal" event={"ID":"eac84a200e5292e5d94479fe1961055a","Type":"ContainerStarted","Data":"47cd6456a9e0cb4f40c0703382223c3c9f54bab8c2b4da96302cf896c7a42b64"}
Jul 13 20:54:04.679015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:04.678594    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-146.ec2.internal" podStartSLOduration=4.678574483 podStartE2EDuration="4.678574483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:04.677806088 +0000 UTC m=+5.616010404" watchObservedRunningTime="2026-07-13 20:54:04.678574483 +0000 UTC m=+5.616778795"
Jul 13 20:54:05.076327 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:05.075593    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:05.076327 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:05.075686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:05.076327 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.075802    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:05.076327 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.075862    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:09.075844231 +0000 UTC m=+10.014048534 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:05.076327 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.076224    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:05.076327 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.076269    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret podName:0420276d-4854-4250-9a5d-5ebdfad81a34 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:09.076253521 +0000 UTC m=+10.014457816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret") pod "global-pull-secret-syncer-2z72g" (UID: "0420276d-4854-4250-9a5d-5ebdfad81a34") : object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:05.277462 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:05.277219    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:05.277462 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.277431    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 20:54:05.277462 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.277452    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 20:54:05.277462 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.277464    2556 projected.go:196] Error preparing data for projected volume kube-api-access-vfs46 for pod openshift-network-diagnostics/network-check-target-j28gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:05.277781 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.277519    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46 podName:d6c59737-a9b2-45fa-8bf6-4691e0d37c77 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:09.277502726 +0000 UTC m=+10.215707033 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfs46" (UniqueName: "kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46") pod "network-check-target-j28gz" (UID: "d6c59737-a9b2-45fa-8bf6-4691e0d37c77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:05.597060 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:05.597028    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:05.597230 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.597167    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:05.597895 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:05.597691    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:05.597895 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:05.597792    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:06.597023 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:06.596476    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:06.597023 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:06.596637    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:07.596952 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:07.596911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:07.597360 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:07.597049    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:07.597360 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:07.597103    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:07.597360 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:07.597220    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:08.597837 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:08.597355    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:08.597837 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:08.597495    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:09.107695 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:09.107646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:09.107881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:09.107705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:09.107881 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.107800    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:09.107881 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.107833    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:09.107881 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.107882    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:17.107859609 +0000 UTC m=+18.046063904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:09.108118 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.107903    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret podName:0420276d-4854-4250-9a5d-5ebdfad81a34 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:17.107893769 +0000 UTC m=+18.046098060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret") pod "global-pull-secret-syncer-2z72g" (UID: "0420276d-4854-4250-9a5d-5ebdfad81a34") : object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:09.309044 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:09.309005    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:09.309209 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.309155    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 20:54:09.309209 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.309185    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 20:54:09.309209 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.309198    2556 projected.go:196] Error preparing data for projected volume kube-api-access-vfs46 for pod openshift-network-diagnostics/network-check-target-j28gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:09.309413 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.309259    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46 podName:d6c59737-a9b2-45fa-8bf6-4691e0d37c77 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:17.309242582 +0000 UTC m=+18.247446872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfs46" (UniqueName: "kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46") pod "network-check-target-j28gz" (UID: "d6c59737-a9b2-45fa-8bf6-4691e0d37c77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:09.597819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:09.597779    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:09.598007 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.597893    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:09.598376 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:09.598250    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:09.598439 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:09.598374    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:10.597507 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:10.597001    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:10.597507 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:10.597157    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:11.597331 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:11.597234    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:11.597331 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:11.597236    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:11.597872 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:11.597387    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:11.597872 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:11.597483    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:12.597283 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:12.597251    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:12.597469 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:12.597385    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:13.597033 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:13.596999    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:13.597189 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:13.597112    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:13.597189 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:13.597175    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:13.597324 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:13.597301    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:14.597298 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:14.597258    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:14.597779 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:14.597430    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:15.596719 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:15.596678    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:15.596888 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:15.596832    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:15.596888 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:15.596884    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:15.596986 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:15.596953    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:16.596928 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:16.596892    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:16.597389 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:16.597033    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:17.163136 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:17.163102    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:17.163136 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:17.163150    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:17.163410 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.163284    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:17.163410 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.163299    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:17.163410 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.163366    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret podName:0420276d-4854-4250-9a5d-5ebdfad81a34 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:33.163332245 +0000 UTC m=+34.101536534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret") pod "global-pull-secret-syncer-2z72g" (UID: "0420276d-4854-4250-9a5d-5ebdfad81a34") : object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:17.163410 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.163388    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:33.163377657 +0000 UTC m=+34.101581950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:17.364791 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:17.364752    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:17.364960 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.364915    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 20:54:17.364960 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.364938    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 20:54:17.364960 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.364951    2556 projected.go:196] Error preparing data for projected volume kube-api-access-vfs46 for pod openshift-network-diagnostics/network-check-target-j28gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:17.365086 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.365016    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46 podName:d6c59737-a9b2-45fa-8bf6-4691e0d37c77 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:33.36499734 +0000 UTC m=+34.303201652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfs46" (UniqueName: "kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46") pod "network-check-target-j28gz" (UID: "d6c59737-a9b2-45fa-8bf6-4691e0d37c77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:17.597349 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:17.597256    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:17.597761 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.597385    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:17.597761 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:17.597433    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:17.597761 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:17.597526    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:18.596746 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:18.596712    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:18.596917 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:18.596835    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:19.598039 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:19.598005    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:19.598431 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:19.598112    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:19.598431 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:19.598200    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:19.598431 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:19.598302    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:20.597236 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:20.597205    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:20.597435 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:20.597333    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:21.597060 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.596781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:21.597868 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.596794    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:21.597868 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:21.597138    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:21.597868 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:21.597199    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:21.712791 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.712763    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9kv8m" event={"ID":"f457d97c-0c11-4379-ac4a-6d8033a4cff7","Type":"ContainerStarted","Data":"5f75f4bb8eda149445053c96875d5988667d5d689f4b3f399068125298dbd8e9"}
Jul 13 20:54:21.714392 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.714371    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbzmm" event={"ID":"6f25a02d-8cc0-406c-a5c1-670ee5d395fb","Type":"ContainerStarted","Data":"d40da6c47fbcfcbbe900be493d4ec5100f60a58b55566e3fc27ab09870a5ffd5"}
Jul 13 20:54:21.718401 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.715568    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9pjnw" event={"ID":"3ba71b3d-1630-4cd4-912c-2ed8f84aad46","Type":"ContainerStarted","Data":"f589259d9ee6f86a10b2b855ad7209a0dadb7db36457aae2205618294a30c52c"}
Jul 13 20:54:21.721021 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.721000    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"d60da95e3b030e1d61f6ea97657b05b10e9386df20ace14004a8a068120ebc09"}
Jul 13 20:54:21.721108 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.721083    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"e25ec6aafaba2d483049b9cd65be3bfc097a37dfe996f5849ead2dba4a731bcc"}
Jul 13 20:54:21.721164 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.721114    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"bf0b9119690f6a64ec13d3e6b1d5dcfb45cacb51fb813d11d1c64ecc7442b3fe"}
Jul 13 20:54:21.721164 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.721150    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"6de4282bc08653f3b551c2ddab73155769219b6117bde64c8f8350ec6ebedb20"}
Jul 13 20:54:21.721248 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.721184    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"d5cfecbf828f2145ca31c7d699c88d1fb2988cf7e94269d55d18cfe5cd089b19"}
Jul 13 20:54:21.722175 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.722152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lv4m5" event={"ID":"21e605a3-44ba-43d8-b5e5-4e365565ed28","Type":"ContainerStarted","Data":"aa63d58ce887284dcf7b9314484885e69edb81cd5d86ffa86e29afb1ba242288"}
Jul 13 20:54:21.723492 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.723472    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj" event={"ID":"1efd37ff-1446-46de-8310-3e3589c2246b","Type":"ContainerStarted","Data":"220e415e4ceb17b209a0dac257666b2aca75681aab60260f14484d36bdc75da3"}
Jul 13 20:54:21.724655 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.724637    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n688c" event={"ID":"38b9c8cd-9d66-430c-b411-e92d0a730deb","Type":"ContainerStarted","Data":"d786dd4af574c90967fcc0cae33b5f5f5b1d043a1acb126d1c0ab3e7e4a01fc2"}
Jul 13 20:54:21.725886 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.725868    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2" containerID="7507a1ba58be48f8913afde53adad51a4abe9efe649d9b1552b127bcd00b5c5f" exitCode=0
Jul 13 20:54:21.725969 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.725894    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerDied","Data":"7507a1ba58be48f8913afde53adad51a4abe9efe649d9b1552b127bcd00b5c5f"}
Jul 13 20:54:21.728564 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.728533    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9kv8m" podStartSLOduration=8.518433648 podStartE2EDuration="22.728521196s" totalImagesPullingTime="14.210087548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:21.728239091 +0000 UTC m=+22.666443403" watchObservedRunningTime="2026-07-13 20:54:21.728521196 +0000 UTC m=+22.666725543"
Jul 13 20:54:21.745064 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.745028    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lv4m5" podStartSLOduration=4.137972857 podStartE2EDuration="22.745017061s" totalImagesPullingTime="18.607044204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:21.744834563 +0000 UTC m=+22.683038876" watchObservedRunningTime="2026-07-13 20:54:21.745017061 +0000 UTC m=+22.683221376"
Jul 13 20:54:21.758838 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.758804    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9pjnw" podStartSLOduration=4.223302803 podStartE2EDuration="22.758793366s" totalImagesPullingTime="18.535490563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:21.758613581 +0000 UTC m=+22.696817893" watchObservedRunningTime="2026-07-13 20:54:21.758793366 +0000 UTC m=+22.696997677"
Jul 13 20:54:21.774937 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:21.774903    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hbzmm" podStartSLOduration=4.224939264 podStartE2EDuration="22.774891742s" totalImagesPullingTime="18.549952478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:21.774434047 +0000 UTC m=+22.712638357" watchObservedRunningTime="2026-07-13 20:54:21.774891742 +0000 UTC m=+22.713096054"
Jul 13 20:54:22.596745 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:22.596711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:22.596904 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:22.596821    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:22.733186 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:22.733152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"7b07d1e7c31ae87b6c7d24122d1239261f511f2b481c77449b4ded5ee90e56c4"}
Jul 13 20:54:22.734833 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:22.734802    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llzq6" event={"ID":"0a0d98d7-d4e9-4395-9ead-400cc91688ac","Type":"ContainerStarted","Data":"a2cc69d3c72b2c3b42d0389bd7bba5dbe381906f3197b61f3982a4b9e1af4cf3"}
Jul 13 20:54:22.751241 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:22.751198    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n688c" podStartSLOduration=5.193988381 podStartE2EDuration="23.75118374s" totalImagesPullingTime="18.557195359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:21.818064971 +0000 UTC m=+22.756269282" watchObservedRunningTime="2026-07-13 20:54:22.75118374 +0000 UTC m=+23.689388114"
Jul 13 20:54:22.751502 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:22.751477    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-llzq6" podStartSLOduration=5.193015107 podStartE2EDuration="23.751469018s" totalImagesPullingTime="18.558453911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:22.75097526 +0000 UTC m=+23.689179572" watchObservedRunningTime="2026-07-13 20:54:22.751469018 +0000 UTC m=+23.689673329"
Jul 13 20:54:22.887060 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:22.887034    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 20:54:23.499396 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:23.499286    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T20:54:22.887055719Z","UUID":"462c1afa-12a2-4e30-8600-3bb6cc9fd907","Handler":null,"Name":"","Endpoint":""}
Jul 13 20:54:23.500772 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:23.500749    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 20:54:23.500909 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:23.500822    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 20:54:23.596774 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:23.596745    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:23.597024 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:23.596868    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:23.597024 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:23.596745    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:23.597024 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:23.596964    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:23.739422 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:23.739381    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj" event={"ID":"1efd37ff-1446-46de-8310-3e3589c2246b","Type":"ContainerStarted","Data":"992937cb2fdf93a861a3f64c4c90ce538b774882627382d021c3c0db7c6275ab"}
Jul 13 20:54:24.596796 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:24.596564    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:24.596976 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:24.596893    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:24.712103 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:24.712024    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:24.713214 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:24.713178    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:24.745425 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:24.745390    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"e872ea0787c54d608133a77434c4d332dcaed5ebc4fbf9fb0883a09579e8a996"}
Jul 13 20:54:24.747657 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:24.747626    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj" event={"ID":"1efd37ff-1446-46de-8310-3e3589c2246b","Type":"ContainerStarted","Data":"12bc5a083a8c36993bc69bca480ff71eb8ada93322f295ec6befbabf6ce96e0f"}
Jul 13 20:54:24.767241 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:24.767187    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hncj" podStartSLOduration=4.305713418 podStartE2EDuration="25.767168681s" totalImagesPullingTime="21.461455263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:24.767014062 +0000 UTC m=+25.705218374" watchObservedRunningTime="2026-07-13 20:54:24.767168681 +0000 UTC m=+25.705373000"
Jul 13 20:54:25.230980 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:25.230947    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:25.231558 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:25.231537    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9kv8m"
Jul 13 20:54:25.597300 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:25.597224    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:25.597481 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:25.597378    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:25.597481 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:25.597378    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:25.597481 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:25.597462    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:26.596916 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.596494    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:26.596916 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:26.596635    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:26.755469 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.755131    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" event={"ID":"9da8359f-85cf-45d3-a81e-82a48d894872","Type":"ContainerStarted","Data":"d95426151496d042e8adc778eee8ceb209476667f38c72f1f07705252c5189c7"}
Jul 13 20:54:26.755469 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.755453    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:26.755660 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.755517    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:26.755660 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.755560    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:26.757197 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.757169    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2" containerID="d644536d7b17b5d3d2db104d87e8c5e81d723e421955f6459419e4b87f7f9897" exitCode=0
Jul 13 20:54:26.757325 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.757235    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerDied","Data":"d644536d7b17b5d3d2db104d87e8c5e81d723e421955f6459419e4b87f7f9897"}
Jul 13 20:54:26.770907 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.770888    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:26.771009 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.771000    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:54:26.784633 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:26.784597    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s" podStartSLOduration=9.166153926 podStartE2EDuration="27.784584945s" totalImagesPullingTime="18.618431019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:26.784426168 +0000 UTC m=+27.722630479" watchObservedRunningTime="2026-07-13 20:54:26.784584945 +0000 UTC m=+27.722789233"
Jul 13 20:54:27.599681 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:27.599655    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:27.600085 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:27.599689    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:27.600085 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:27.599747    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:27.600085 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:27.599893    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:28.596770 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.596739    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:28.596917 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:28.596893    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:28.762686 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.762655    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j28gz"]
Jul 13 20:54:28.763108 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.762766    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:28.763108 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:28.762836    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:28.764982 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.764959    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-grcrp"]
Jul 13 20:54:28.765655 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.765629    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2" containerID="d322a168e2c15155fd511f74f6e8177fbcbecb863e6cbcf592d945e98b35c0f9" exitCode=0
Jul 13 20:54:28.765759 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.765698    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:28.765759 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.765709    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerDied","Data":"d322a168e2c15155fd511f74f6e8177fbcbecb863e6cbcf592d945e98b35c0f9"}
Jul 13 20:54:28.765868 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.765761    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z72g"]
Jul 13 20:54:28.765868 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:28.765817    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:28.765962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:28.765947    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:28.766059 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:28.766032    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:30.596950 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:30.596756    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:30.597494 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:30.596804    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:30.597494 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:30.597056    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:30.597494 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:30.596818    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:30.597494 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:30.597113    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:30.597494 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:30.597148    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:30.772924 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:30.772888    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2" containerID="94f06e33ba85d4fc4dfae407e97cbbf1bd1e76cdb8f21e9708d93119c9760f28" exitCode=0
Jul 13 20:54:30.773079 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:30.772948    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerDied","Data":"94f06e33ba85d4fc4dfae407e97cbbf1bd1e76cdb8f21e9708d93119c9760f28"}
Jul 13 20:54:32.596828 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:32.596794    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:32.597482 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:32.596800    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:32.597482 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:32.596910    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:32.597482 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:32.596999    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:32.597482 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:32.596804    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:32.597482 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:32.597110    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:33.186626 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:33.186592    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:33.186837 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:33.186646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:33.186837 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.186760    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:33.186837 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.186832    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:55:05.186814504 +0000 UTC m=+66.125018805 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 20:54:33.187003 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.186764    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:33.187003 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.186900    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret podName:0420276d-4854-4250-9a5d-5ebdfad81a34 nodeName:}" failed. No retries permitted until 2026-07-13 20:55:05.186886865 +0000 UTC m=+66.125091154 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret") pod "global-pull-secret-syncer-2z72g" (UID: "0420276d-4854-4250-9a5d-5ebdfad81a34") : object "kube-system"/"original-pull-secret" not registered
Jul 13 20:54:33.388303 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:33.388264    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:33.388475 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.388422    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 20:54:33.388475 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.388446    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 20:54:33.388475 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.388459    2556 projected.go:196] Error preparing data for projected volume kube-api-access-vfs46 for pod openshift-network-diagnostics/network-check-target-j28gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:33.388631 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:33.388522    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46 podName:d6c59737-a9b2-45fa-8bf6-4691e0d37c77 nodeName:}" failed. No retries permitted until 2026-07-13 20:55:05.388506115 +0000 UTC m=+66.326710407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vfs46" (UniqueName: "kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46") pod "network-check-target-j28gz" (UID: "d6c59737-a9b2-45fa-8bf6-4691e0d37c77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 20:54:34.597010 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.596974    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:34.597443 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.596976    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:34.597443 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:34.597098    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z72g" podUID="0420276d-4854-4250-9a5d-5ebdfad81a34"
Jul 13 20:54:34.597443 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.597124    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:34.597443 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:34.597234    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:54:34.597443 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:34.597317    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j28gz" podUID="d6c59737-a9b2-45fa-8bf6-4691e0d37c77"
Jul 13 20:54:34.858853 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.858787    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-146.ec2.internal" event="NodeReady"
Jul 13 20:54:34.858982 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.858915    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 20:54:34.903422 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.903390    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4fxfq"]
Jul 13 20:54:34.918377 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.918352    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7gknj"]
Jul 13 20:54:34.918539 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.918444    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:34.921464 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.921436    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 20:54:34.921601 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.921470    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 20:54:34.921601 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.921566    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f67kb\""
Jul 13 20:54:34.932330 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.932313    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7gknj"]
Jul 13 20:54:34.932421 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.932351    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4fxfq"]
Jul 13 20:54:34.932421 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.932412    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:34.934782 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.934760    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 20:54:34.935052 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.935032    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q596p\""
Jul 13 20:54:34.935283 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.935264    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 20:54:34.935412 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:34.935315    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 20:54:35.000257 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.000224    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/326c721f-7a82-4669-b58c-153c8ac68f50-config-volume\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.000257 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.000258    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvtfz\" (UniqueName: \"kubernetes.io/projected/326c721f-7a82-4669-b58c-153c8ac68f50-kube-api-access-qvtfz\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.000468 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.000315    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.000468 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.000389    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20d69c93-0f84-46ed-95ab-2d19b1515e04-cert\") pod \"ingress-canary-7gknj\" (UID: \"20d69c93-0f84-46ed-95ab-2d19b1515e04\") " pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:35.000468 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.000415    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsg2m\" (UniqueName: \"kubernetes.io/projected/20d69c93-0f84-46ed-95ab-2d19b1515e04-kube-api-access-hsg2m\") pod \"ingress-canary-7gknj\" (UID: \"20d69c93-0f84-46ed-95ab-2d19b1515e04\") " pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:35.000468 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.000445    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/326c721f-7a82-4669-b58c-153c8ac68f50-tmp-dir\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.101526 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.101489    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/326c721f-7a82-4669-b58c-153c8ac68f50-tmp-dir\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.101704 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.101537    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/326c721f-7a82-4669-b58c-153c8ac68f50-config-volume\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.101704 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.101564    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvtfz\" (UniqueName: \"kubernetes.io/projected/326c721f-7a82-4669-b58c-153c8ac68f50-kube-api-access-qvtfz\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.101823 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.101704    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.101823 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.101750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20d69c93-0f84-46ed-95ab-2d19b1515e04-cert\") pod \"ingress-canary-7gknj\" (UID: \"20d69c93-0f84-46ed-95ab-2d19b1515e04\") " pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:35.101823 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.101778    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsg2m\" (UniqueName: \"kubernetes.io/projected/20d69c93-0f84-46ed-95ab-2d19b1515e04-kube-api-access-hsg2m\") pod \"ingress-canary-7gknj\" (UID: \"20d69c93-0f84-46ed-95ab-2d19b1515e04\") " pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:35.101967 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:35.101852    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:54:35.101967 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.101865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/326c721f-7a82-4669-b58c-153c8ac68f50-tmp-dir\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.101967 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:35.101930    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:35.601908199 +0000 UTC m=+36.540112515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:54:35.102221 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.102200    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/326c721f-7a82-4669-b58c-153c8ac68f50-config-volume\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.105901 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.105777    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20d69c93-0f84-46ed-95ab-2d19b1515e04-cert\") pod \"ingress-canary-7gknj\" (UID: \"20d69c93-0f84-46ed-95ab-2d19b1515e04\") " pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:35.110072 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.110002    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvtfz\" (UniqueName: \"kubernetes.io/projected/326c721f-7a82-4669-b58c-153c8ac68f50-kube-api-access-qvtfz\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.110371 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.110333    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsg2m\" (UniqueName: \"kubernetes.io/projected/20d69c93-0f84-46ed-95ab-2d19b1515e04-kube-api-access-hsg2m\") pod \"ingress-canary-7gknj\" (UID: \"20d69c93-0f84-46ed-95ab-2d19b1515e04\") " pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:35.240474 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.240456    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7gknj"
Jul 13 20:54:35.424709 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.424675    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7gknj"]
Jul 13 20:54:35.428291 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:54:35.428267    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d69c93_0f84_46ed_95ab_2d19b1515e04.slice/crio-288d4cdd5e3d9f4d0807c4449a004b9c1f7f0f44a57a598118309ab2b3cc569a WatchSource:0}: Error finding container 288d4cdd5e3d9f4d0807c4449a004b9c1f7f0f44a57a598118309ab2b3cc569a: Status 404 returned error can't find the container with id 288d4cdd5e3d9f4d0807c4449a004b9c1f7f0f44a57a598118309ab2b3cc569a
Jul 13 20:54:35.605392 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.605361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:35.605757 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:35.605495    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:54:35.605757 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:35.605549    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:36.605532829 +0000 UTC m=+37.543737136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:54:35.788892 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.788813    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2" containerID="71abe176063a4e482e8fb0610138b844bc8b8fc3377ce347be7c1a63f69db56c" exitCode=0
Jul 13 20:54:35.789034 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.788892    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerDied","Data":"71abe176063a4e482e8fb0610138b844bc8b8fc3377ce347be7c1a63f69db56c"}
Jul 13 20:54:35.790271 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:35.790040    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7gknj" event={"ID":"20d69c93-0f84-46ed-95ab-2d19b1515e04","Type":"ContainerStarted","Data":"288d4cdd5e3d9f4d0807c4449a004b9c1f7f0f44a57a598118309ab2b3cc569a"}
Jul 13 20:54:36.597257 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.597041    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:54:36.597466 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.597086    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:54:36.597466 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.597086    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:54:36.599790 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.599766    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n7pmh\""
Jul 13 20:54:36.599917 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.599766    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 20:54:36.600686 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.600665    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 20:54:36.600783 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.600720    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 20:54:36.600925 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.600907    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-plnw6\""
Jul 13 20:54:36.600998 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.600984    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 20:54:36.614599 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.614575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:36.614952 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:36.614726    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:54:36.614952 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:36.614789    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:38.614773306 +0000 UTC m=+39.552977596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:54:36.796328 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.796293    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2" containerID="6c59b4a28a4ce99c42b4f6ed95f790e9d93e7caa933847596564bd66689cd816" exitCode=0
Jul 13 20:54:36.796514 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:36.796377    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerDied","Data":"6c59b4a28a4ce99c42b4f6ed95f790e9d93e7caa933847596564bd66689cd816"}
Jul 13 20:54:37.802692 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:37.802658    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" event={"ID":"fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2","Type":"ContainerStarted","Data":"e58719415c789bedde399f835bc63d9a6ac50e46b143e16bc3bdf01209c6f29f"}
Jul 13 20:54:37.804090 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:37.804061    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7gknj" event={"ID":"20d69c93-0f84-46ed-95ab-2d19b1515e04","Type":"ContainerStarted","Data":"2aa513e9c140b5bdfb228a79e66f83d69d02b490eef7a9c7f74f17fe199bd063"}
Jul 13 20:54:37.827390 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:37.827290    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zmwk4" podStartSLOduration=8.364735238 podStartE2EDuration="38.827275116s" totalImagesPullingTime="30.462539878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:37.826170996 +0000 UTC m=+38.764375297" watchObservedRunningTime="2026-07-13 20:54:37.827275116 +0000 UTC m=+38.765479427"
Jul 13 20:54:37.842317 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:37.842278    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7gknj" podStartSLOduration=1.703804943 podStartE2EDuration="3.842265583s" totalImagesPullingTime="2.13846064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:54:37.841511551 +0000 UTC m=+38.779715877" watchObservedRunningTime="2026-07-13 20:54:37.842265583 +0000 UTC m=+38.780469920"
Jul 13 20:54:38.631521 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:38.631494    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:38.631678 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:38.631609    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:54:38.631678 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:38.631664    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:42.631648719 +0000 UTC m=+43.569853007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:54:42.659886 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:42.659849    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:42.660411 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:42.659968    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:54:42.660411 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:42.660021    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:54:50.660007006 +0000 UTC m=+51.598211296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:54:50.716654 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:50.716615    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:54:50.717107 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:50.716760    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:54:50.717107 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:54:50.716822    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:55:06.716807179 +0000 UTC m=+67.655011485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:54:58.777020 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:54:58.776994    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qwm4s"
Jul 13 20:55:05.209610 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.209559    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:55:05.209610 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.209616    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:55:05.212495 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.212472    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 20:55:05.212613 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.212508    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 20:55:05.220726 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:55:05.220703    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 13 20:55:05.220790 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:55:05.220778    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:56:09.220758562 +0000 UTC m=+130.158962869 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : secret "metrics-daemon-secret" not found
Jul 13 20:55:05.224200 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.224169    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0420276d-4854-4250-9a5d-5ebdfad81a34-original-pull-secret\") pod \"global-pull-secret-syncer-2z72g\" (UID: \"0420276d-4854-4250-9a5d-5ebdfad81a34\") " pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:55:05.411128 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.411080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:55:05.414095 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.414072    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 20:55:05.420423 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.420405    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z72g"
Jul 13 20:55:05.423804 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.423787    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 20:55:05.434576 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.434553    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfs46\" (UniqueName: \"kubernetes.io/projected/d6c59737-a9b2-45fa-8bf6-4691e0d37c77-kube-api-access-vfs46\") pod \"network-check-target-j28gz\" (UID: \"d6c59737-a9b2-45fa-8bf6-4691e0d37c77\") " pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:55:05.539064 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.539024    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z72g"]
Jul 13 20:55:05.542983 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:55:05.542952    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0420276d_4854_4250_9a5d_5ebdfad81a34.slice/crio-6a677c3e810ee7da09d660729807bce5a04be91216c71d9c6775f487c8ad01ec WatchSource:0}: Error finding container 6a677c3e810ee7da09d660729807bce5a04be91216c71d9c6775f487c8ad01ec: Status 404 returned error can't find the container with id 6a677c3e810ee7da09d660729807bce5a04be91216c71d9c6775f487c8ad01ec
Jul 13 20:55:05.711015 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.710985    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n7pmh\""
Jul 13 20:55:05.719465 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.719378    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:55:05.836348 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.836306    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j28gz"]
Jul 13 20:55:05.839510 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:55:05.839487    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c59737_a9b2_45fa_8bf6_4691e0d37c77.slice/crio-7655e186e03b8f4b7d58a603b50a08f1b4653ee62934cf839812335baf1f1cb1 WatchSource:0}: Error finding container 7655e186e03b8f4b7d58a603b50a08f1b4653ee62934cf839812335baf1f1cb1: Status 404 returned error can't find the container with id 7655e186e03b8f4b7d58a603b50a08f1b4653ee62934cf839812335baf1f1cb1
Jul 13 20:55:05.871980 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.871948    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j28gz" event={"ID":"d6c59737-a9b2-45fa-8bf6-4691e0d37c77","Type":"ContainerStarted","Data":"7655e186e03b8f4b7d58a603b50a08f1b4653ee62934cf839812335baf1f1cb1"}
Jul 13 20:55:05.872959 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:05.872934    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z72g" event={"ID":"0420276d-4854-4250-9a5d-5ebdfad81a34","Type":"ContainerStarted","Data":"6a677c3e810ee7da09d660729807bce5a04be91216c71d9c6775f487c8ad01ec"}
Jul 13 20:55:06.720056 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:06.720010    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:55:06.720523 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:55:06.720189    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:55:06.720523 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:55:06.720275    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:55:38.720253081 +0000 UTC m=+99.658457375 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:55:10.886295 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:10.886254    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j28gz" event={"ID":"d6c59737-a9b2-45fa-8bf6-4691e0d37c77","Type":"ContainerStarted","Data":"2edb56bdcb6b945e2de95ca3e2f95cc83903e8e0174ddd469228516bef88908a"}
Jul 13 20:55:10.886752 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:10.886366    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:55:10.887810 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:10.887784    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z72g" event={"ID":"0420276d-4854-4250-9a5d-5ebdfad81a34","Type":"ContainerStarted","Data":"003693e6f4ee702289274ffed2293d33601b8b0c25c84a2277b0c4447df87dbd"}
Jul 13 20:55:10.904209 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:10.904169    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j28gz" podStartSLOduration=67.432814829 podStartE2EDuration="1m11.904157048s" totalImagesPullingTime="4.471342219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:55:10.902972524 +0000 UTC m=+71.841176846" watchObservedRunningTime="2026-07-13 20:55:10.904157048 +0000 UTC m=+71.842361359"
Jul 13 20:55:38.731056 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:38.731020    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:55:38.731483 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:55:38.731137    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:55:38.731483 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:55:38.731201    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:56:42.731185444 +0000 UTC m=+163.669389733 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:55:41.892097 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:41.892068    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j28gz"
Jul 13 20:55:41.907305 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:55:41.907256    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2z72g" podStartSLOduration=98.135982528 podStartE2EDuration="1m42.907242654s" totalImagesPullingTime="4.771260126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:55:10.921118791 +0000 UTC m=+71.859323103" watchObservedRunningTime="2026-07-13 20:55:41.907242654 +0000 UTC m=+102.845446965"
Jul 13 20:56:09.237020 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:56:09.236977    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:56:09.237516 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:56:09.237122    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 13 20:56:09.237516 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:56:09.237196    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:11.237179767 +0000 UTC m=+252.175384056 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : secret "metrics-daemon-secret" not found
Jul 13 20:56:37.928594 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:56:37.928554    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4fxfq" podUID="326c721f-7a82-4669-b58c-153c8ac68f50"
Jul 13 20:56:38.086355 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:56:38.086309    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:56:39.615587 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:56:39.615549    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:56:42.755102 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:56:42.755058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:56:42.755519 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:56:42.755185    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 20:56:42.755519 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:56:42.755239    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls podName:326c721f-7a82-4669-b58c-153c8ac68f50 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:44.755224524 +0000 UTC m=+285.693428813 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls") pod "dns-default-4fxfq" (UID: "326c721f-7a82-4669-b58c-153c8ac68f50") : secret "dns-default-metrics-tls" not found
Jul 13 20:56:53.597482 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:56:53.597388    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:58:04.143731 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.143696    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-mvx2z"]
Jul 13 20:58:04.145510 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.145491    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.151657 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.151637    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 13 20:58:04.152439 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.152423    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 13 20:58:04.152717 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.152699    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zhtlt\""
Jul 13 20:58:04.153544 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.153525    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 20:58:04.153826 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.153811    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 20:58:04.154539 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.154522    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"]
Jul 13 20:58:04.156262 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.156249    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.158055 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.158038    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 13 20:58:04.159220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.159204    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 13 20:58:04.159301 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.159207    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 13 20:58:04.159505 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.159490    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 13 20:58:04.159584 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.159538    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8vskr\""
Jul 13 20:58:04.162616 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.162599    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bd4c58db5-r9djk"]
Jul 13 20:58:04.163133 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.163116    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 13 20:58:04.164197 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.164181    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7488d7cbb8-nn48c"]
Jul 13 20:58:04.164299 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.164286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.165832 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.165817    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"]
Jul 13 20:58:04.165925 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.165918    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.167515 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.167499    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.168186 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.168172    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 13 20:58:04.168270 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.168190    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 13 20:58:04.168458 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.168430    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 13 20:58:04.168544 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.168446    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bwbzd\""
Jul 13 20:58:04.173617 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173600    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-serving-cert\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.173617 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173612    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 20:58:04.173763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173624    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.173763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173642    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.173763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173657    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41e2a709-c033-4b04-99ef-c5876668a5b1-ca-trust-extracted\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.173763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173682    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-bound-sa-token\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.173763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173715    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-certificates\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.173763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173726    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 13 20:58:04.173763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173753    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-installation-pull-secrets\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173809    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-image-registry-private-configuration\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173854    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173872    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-snapshots\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173900    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trqlq\" (UniqueName: \"kubernetes.io/projected/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-kube-api-access-trqlq\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173923    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7vhd\" (UniqueName: \"kubernetes.io/projected/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-kube-api-access-k7vhd\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173947    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn5xb\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-kube-api-access-gn5xb\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.173975    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-tmp\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.174065 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.174029    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.174369 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.174070    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-trusted-ca\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.174369 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.174082    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 13 20:58:04.174875 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.174856    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 20:58:04.174932 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.174881    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 13 20:58:04.175162 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175140    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 13 20:58:04.175235 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175169    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 13 20:58:04.175235 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175183    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-z6f9j\""
Jul 13 20:58:04.175235 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175191    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b7n2m\""
Jul 13 20:58:04.175404 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175150    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 13 20:58:04.175404 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175253    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 20:58:04.175404 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175267    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 20:58:04.175529 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.175516    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 13 20:58:04.190016 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.189998    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"]
Jul 13 20:58:04.190758 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.190714    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"]
Jul 13 20:58:04.191408 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.191394    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7488d7cbb8-nn48c"]
Jul 13 20:58:04.191982 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.191968    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bd4c58db5-r9djk"]
Jul 13 20:58:04.214168 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.214134    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-mvx2z"]
Jul 13 20:58:04.240406 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.240380    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"]
Jul 13 20:58:04.242110 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.242095    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"
Jul 13 20:58:04.244271 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.244251    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"]
Jul 13 20:58:04.244864 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.244848    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-blsj5\""
Jul 13 20:58:04.245848 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.245828    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-gd4wz"]
Jul 13 20:58:04.245980 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.245966    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"
Jul 13 20:58:04.247244 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.247230    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"]
Jul 13 20:58:04.247375 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.247361    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.248606 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.248592    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 13 20:58:04.248701 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.248677    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4rcc4\""
Jul 13 20:58:04.248780 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.248767    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 13 20:58:04.248893 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.248878    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:04.249982 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.249964    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p2jmh\""
Jul 13 20:58:04.250078 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.250059    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 13 20:58:04.250452 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.250438    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 13 20:58:04.251624 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.251605    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 13 20:58:04.251721 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.251676    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 13 20:58:04.251721 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.251683    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 13 20:58:04.251721 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.251683    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 13 20:58:04.252168 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.252138    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 13 20:58:04.252244 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.252146    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hpgt9\""
Jul 13 20:58:04.255595 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.255577    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"]
Jul 13 20:58:04.260182 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.260159    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 13 20:58:04.260651 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.260634    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-gd4wz"]
Jul 13 20:58:04.261888 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.261858    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"]
Jul 13 20:58:04.269352 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.269312    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"]
Jul 13 20:58:04.274760 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274742    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-snapshots\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.274849 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274767    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7vhd\" (UniqueName: \"kubernetes.io/projected/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-kube-api-access-k7vhd\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.274849 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274788    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0cd5c58a-b847-450e-aebd-3a21b97e4f10-trusted-ca\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.274849 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-stats-auth\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.274849 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274825    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-tmp\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.275037 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274859    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-bound-sa-token\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.275037 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.275037 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274914    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cd5c58a-b847-450e-aebd-3a21b97e4f10-config\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.275037 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.274946    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-serving-cert\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.275037 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275029    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8dsp\" (UniqueName: \"kubernetes.io/projected/f0172f3d-38f3-4259-88f8-258f6424f8e0-kube-api-access-l8dsp\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275104    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-tmp\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.275157    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.275169    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd4c58db5-r9djk: secret "image-registry-tls" not found
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.275210    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls podName:41e2a709-c033-4b04-99ef-c5876668a5b1 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:04.775191717 +0000 UTC m=+245.713396010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls") pod "image-registry-6bd4c58db5-r9djk" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1") : secret "image-registry-tls" not found
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275227    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.275263 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275257    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275298    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzktz\" (UniqueName: \"kubernetes.io/projected/0cd5c58a-b847-450e-aebd-3a21b97e4f10-kube-api-access-bzktz\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275332    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-installation-pull-secrets\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275377    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-default-certificate\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275414    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-image-registry-private-configuration\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275436    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-snapshots\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275459    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-trusted-ca\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275489    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275516    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfwl8\" (UniqueName: \"kubernetes.io/projected/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-kube-api-access-jfwl8\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275544    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trqlq\" (UniqueName: \"kubernetes.io/projected/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-kube-api-access-trqlq\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275567    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn5xb\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-kube-api-access-gn5xb\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275590    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275622    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cd5c58a-b847-450e-aebd-3a21b97e4f10-serving-cert\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275658    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41e2a709-c033-4b04-99ef-c5876668a5b1-ca-trust-extracted\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.275744 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275730    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-727zn\" (UniqueName: \"kubernetes.io/projected/77b67b18-31e2-4feb-9ea2-fe4fa51056f3-kube-api-access-727zn\") pod \"network-check-source-869dcfcdcd-8b78v\" (UID: \"77b67b18-31e2-4feb-9ea2-fe4fa51056f3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"
Jul 13 20:58:04.276476 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275771    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn2gb\" (UniqueName: \"kubernetes.io/projected/90d21d66-81d6-464c-a857-7d43f52beb1d-kube-api-access-vn2gb\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.276476 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275803    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90d21d66-81d6-464c-a857-7d43f52beb1d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.276476 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275829    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:04.276476 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-certificates\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.276476 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.275882    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nhw9\" (UniqueName: \"kubernetes.io/projected/97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2-kube-api-access-8nhw9\") pod \"volume-data-source-validator-78495597f6-nqddc\" (UID: \"97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"
Jul 13 20:58:04.276476 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.276028    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.276476 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.276125    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.276830 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.276593    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41e2a709-c033-4b04-99ef-c5876668a5b1-ca-trust-extracted\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.276830 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.276650    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.276992 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.276970    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-trusted-ca\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.277146 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.277128    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-certificates\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.277819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.277796    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-serving-cert\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.277958 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.277938    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.278112 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.278056    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-image-registry-private-configuration\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.278396 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.278381    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-installation-pull-secrets\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.287939 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.287913    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-bound-sa-token\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.288007 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.287952    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn5xb\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-kube-api-access-gn5xb\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.288465 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.288451    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7vhd\" (UniqueName: \"kubernetes.io/projected/7518b6cb-fdc2-46b6-934c-e73079d7a0d2-kube-api-access-k7vhd\") pod \"insights-operator-68d9dd94cf-mvx2z\" (UID: \"7518b6cb-fdc2-46b6-934c-e73079d7a0d2\") " pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.288511 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.288486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trqlq\" (UniqueName: \"kubernetes.io/projected/f7e89441-6447-4b44-b1e3-b3a8ead9aa13-kube-api-access-trqlq\") pod \"kube-storage-version-migrator-operator-799cb7f958-bkql9\" (UID: \"f7e89441-6447-4b44-b1e3-b3a8ead9aa13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.376429 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376389    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.376429 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376431    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cd5c58a-b847-450e-aebd-3a21b97e4f10-config\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.376659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376459    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8dsp\" (UniqueName: \"kubernetes.io/projected/f0172f3d-38f3-4259-88f8-258f6424f8e0-kube-api-access-l8dsp\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.376659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376501    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.376659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376528    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzktz\" (UniqueName: \"kubernetes.io/projected/0cd5c58a-b847-450e-aebd-3a21b97e4f10-kube-api-access-bzktz\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.376659 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.376559    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:04.87653869 +0000 UTC m=+245.814742982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : configmap references non-existent config key: service-ca.crt
Jul 13 20:58:04.376659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376591    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-default-certificate\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.376659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376630    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwl8\" (UniqueName: \"kubernetes.io/projected/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-kube-api-access-jfwl8\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376664    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376700    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cd5c58a-b847-450e-aebd-3a21b97e4f10-serving-cert\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.376633    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376754    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-727zn\" (UniqueName: \"kubernetes.io/projected/77b67b18-31e2-4feb-9ea2-fe4fa51056f3-kube-api-access-727zn\") pod \"network-check-source-869dcfcdcd-8b78v\" (UID: \"77b67b18-31e2-4feb-9ea2-fe4fa51056f3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.376787    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:04.876769233 +0000 UTC m=+245.814973542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : secret "router-metrics-certs-default" not found
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.376785    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376807    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn2gb\" (UniqueName: \"kubernetes.io/projected/90d21d66-81d6-464c-a857-7d43f52beb1d-kube-api-access-vn2gb\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90d21d66-81d6-464c-a857-7d43f52beb1d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.376870    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls podName:90d21d66-81d6-464c-a857-7d43f52beb1d nodeName:}" failed. No retries permitted until 2026-07-13 20:58:04.876836908 +0000 UTC m=+245.815041211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2vxxv" (UID: "90d21d66-81d6-464c-a857-7d43f52beb1d") : secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376905    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:04.376962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376955    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nhw9\" (UniqueName: \"kubernetes.io/projected/97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2-kube-api-access-8nhw9\") pod \"volume-data-source-validator-78495597f6-nqddc\" (UID: \"97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"
Jul 13 20:58:04.377729 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.376996    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0cd5c58a-b847-450e-aebd-3a21b97e4f10-trusted-ca\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.377729 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.377011    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 20:58:04.377729 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.377018    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-stats-auth\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.377729 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.377054    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls podName:b9dfd806-8ff8-4ff5-b845-ae9eba86ab64 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:04.877041818 +0000 UTC m=+245.815246110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls") pod "cluster-samples-operator-65db858894-th7z9" (UID: "b9dfd806-8ff8-4ff5-b845-ae9eba86ab64") : secret "samples-operator-tls" not found
Jul 13 20:58:04.377729 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.377368    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cd5c58a-b847-450e-aebd-3a21b97e4f10-config\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.377729 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.377569    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90d21d66-81d6-464c-a857-7d43f52beb1d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.378004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.377888    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0cd5c58a-b847-450e-aebd-3a21b97e4f10-trusted-ca\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.379087 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.379061    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cd5c58a-b847-450e-aebd-3a21b97e4f10-serving-cert\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.379292 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.379273    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-stats-auth\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.379615 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.379600    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-default-certificate\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.391800 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.391774    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-727zn\" (UniqueName: \"kubernetes.io/projected/77b67b18-31e2-4feb-9ea2-fe4fa51056f3-kube-api-access-727zn\") pod \"network-check-source-869dcfcdcd-8b78v\" (UID: \"77b67b18-31e2-4feb-9ea2-fe4fa51056f3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"
Jul 13 20:58:04.391899 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.391859    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn2gb\" (UniqueName: \"kubernetes.io/projected/90d21d66-81d6-464c-a857-7d43f52beb1d-kube-api-access-vn2gb\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.392449 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.392430    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfwl8\" (UniqueName: \"kubernetes.io/projected/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-kube-api-access-jfwl8\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:04.392560 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.392543    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzktz\" (UniqueName: \"kubernetes.io/projected/0cd5c58a-b847-450e-aebd-3a21b97e4f10-kube-api-access-bzktz\") pod \"console-operator-7866497494-gd4wz\" (UID: \"0cd5c58a-b847-450e-aebd-3a21b97e4f10\") " pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.395443 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.395405    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nhw9\" (UniqueName: \"kubernetes.io/projected/97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2-kube-api-access-8nhw9\") pod \"volume-data-source-validator-78495597f6-nqddc\" (UID: \"97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"
Jul 13 20:58:04.400616 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.400601    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8dsp\" (UniqueName: \"kubernetes.io/projected/f0172f3d-38f3-4259-88f8-258f6424f8e0-kube-api-access-l8dsp\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.453492 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.453462    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z"
Jul 13 20:58:04.464386 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.464361    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"
Jul 13 20:58:04.552899 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.552871    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"
Jul 13 20:58:04.559930 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.559894    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"
Jul 13 20:58:04.568193 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.568132    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:04.587682 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.587560    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-mvx2z"]
Jul 13 20:58:04.590655 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:04.590622    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7518b6cb_fdc2_46b6_934c_e73079d7a0d2.slice/crio-25e534f7a69e596517a36b9f6600924b8ce0736b2d6143232ed1917d616bddeb WatchSource:0}: Error finding container 25e534f7a69e596517a36b9f6600924b8ce0736b2d6143232ed1917d616bddeb: Status 404 returned error can't find the container with id 25e534f7a69e596517a36b9f6600924b8ce0736b2d6143232ed1917d616bddeb
Jul 13 20:58:04.604305 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.604278    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9"]
Jul 13 20:58:04.608949 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:04.608924    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e89441_6447_4b44_b1e3_b3a8ead9aa13.slice/crio-ec748e803ff80ef676c72907ba77244a2078093ad53bc534efea9b226670dad9 WatchSource:0}: Error finding container ec748e803ff80ef676c72907ba77244a2078093ad53bc534efea9b226670dad9: Status 404 returned error can't find the container with id ec748e803ff80ef676c72907ba77244a2078093ad53bc534efea9b226670dad9
Jul 13 20:58:04.702196 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.702162    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v"]
Jul 13 20:58:04.703915 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:04.703877    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b67b18_31e2_4feb_9ea2_fe4fa51056f3.slice/crio-22399c4c5d3fdbc93ad4246e0b1566f2eeb548854b7789ae0db78a9ad723cca9 WatchSource:0}: Error finding container 22399c4c5d3fdbc93ad4246e0b1566f2eeb548854b7789ae0db78a9ad723cca9: Status 404 returned error can't find the container with id 22399c4c5d3fdbc93ad4246e0b1566f2eeb548854b7789ae0db78a9ad723cca9
Jul 13 20:58:04.779593 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.779564    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:04.779737 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.779702    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 20:58:04.779737 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.779717    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd4c58db5-r9djk: secret "image-registry-tls" not found
Jul 13 20:58:04.779848 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.779759    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls podName:41e2a709-c033-4b04-99ef-c5876668a5b1 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:05.779746864 +0000 UTC m=+246.717951156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls") pod "image-registry-6bd4c58db5-r9djk" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1") : secret "image-registry-tls" not found
Jul 13 20:58:04.880106 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.880070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.880280 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.880117    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:04.880280 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.880165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:04.880280 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.880196    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:04.880280 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.880217    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 20:58:04.880449 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.880290    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 20:58:04.880449 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.880294    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:04.880449 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.880298    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:05.880278306 +0000 UTC m=+246.818482606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : secret "router-metrics-certs-default" not found
Jul 13 20:58:04.880449 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.880407    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls podName:b9dfd806-8ff8-4ff5-b845-ae9eba86ab64 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:05.880386246 +0000 UTC m=+246.818590540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls") pod "cluster-samples-operator-65db858894-th7z9" (UID: "b9dfd806-8ff8-4ff5-b845-ae9eba86ab64") : secret "samples-operator-tls" not found
Jul 13 20:58:04.880449 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.880423    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:05.880414129 +0000 UTC m=+246.818618428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : configmap references non-existent config key: service-ca.crt
Jul 13 20:58:04.880449 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:04.880439    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls podName:90d21d66-81d6-464c-a857-7d43f52beb1d nodeName:}" failed. No retries permitted until 2026-07-13 20:58:05.880431646 +0000 UTC m=+246.818635934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2vxxv" (UID: "90d21d66-81d6-464c-a857-7d43f52beb1d") : secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:04.923362 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.923260    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc"]
Jul 13 20:58:04.927219 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:04.927189    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-gd4wz"]
Jul 13 20:58:04.927829 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:04.927806    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d7cdc8_2942_4d1a_a64e_aaef2f94dbc2.slice/crio-f4d252948e233737a112d85ebd2c96a0df77eadfaaa70cb551de9350e92b4140 WatchSource:0}: Error finding container f4d252948e233737a112d85ebd2c96a0df77eadfaaa70cb551de9350e92b4140: Status 404 returned error can't find the container with id f4d252948e233737a112d85ebd2c96a0df77eadfaaa70cb551de9350e92b4140
Jul 13 20:58:04.930858 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:04.930836    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd5c58a_b847_450e_aebd_3a21b97e4f10.slice/crio-a78b9c620323c1b374e4b8b06350cdf3e922fd8e0a94f828dd782157e0e2ce36 WatchSource:0}: Error finding container a78b9c620323c1b374e4b8b06350cdf3e922fd8e0a94f828dd782157e0e2ce36: Status 404 returned error can't find the container with id a78b9c620323c1b374e4b8b06350cdf3e922fd8e0a94f828dd782157e0e2ce36
Jul 13 20:58:05.286983 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.286873    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc" event={"ID":"97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2","Type":"ContainerStarted","Data":"f4d252948e233737a112d85ebd2c96a0df77eadfaaa70cb551de9350e92b4140"}
Jul 13 20:58:05.288141 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.288112    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gd4wz" event={"ID":"0cd5c58a-b847-450e-aebd-3a21b97e4f10","Type":"ContainerStarted","Data":"a78b9c620323c1b374e4b8b06350cdf3e922fd8e0a94f828dd782157e0e2ce36"}
Jul 13 20:58:05.289555 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.289506    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9" event={"ID":"f7e89441-6447-4b44-b1e3-b3a8ead9aa13","Type":"ContainerStarted","Data":"ec748e803ff80ef676c72907ba77244a2078093ad53bc534efea9b226670dad9"}
Jul 13 20:58:05.290802 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.290759    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z" event={"ID":"7518b6cb-fdc2-46b6-934c-e73079d7a0d2","Type":"ContainerStarted","Data":"25e534f7a69e596517a36b9f6600924b8ce0736b2d6143232ed1917d616bddeb"}
Jul 13 20:58:05.292387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.292299    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v" event={"ID":"77b67b18-31e2-4feb-9ea2-fe4fa51056f3","Type":"ContainerStarted","Data":"f5539817bf7032512e61f45d7fbec535034f53840dd34e30cfe2cb948d79addb"}
Jul 13 20:58:05.292387 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.292360    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v" event={"ID":"77b67b18-31e2-4feb-9ea2-fe4fa51056f3","Type":"ContainerStarted","Data":"22399c4c5d3fdbc93ad4246e0b1566f2eeb548854b7789ae0db78a9ad723cca9"}
Jul 13 20:58:05.308145 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.307959    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8b78v" podStartSLOduration=1.307943035 podStartE2EDuration="1.307943035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:05.306523746 +0000 UTC m=+246.244728057" watchObservedRunningTime="2026-07-13 20:58:05.307943035 +0000 UTC m=+246.246147347"
Jul 13 20:58:05.787657 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.787584    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:05.787846 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.787820    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 20:58:05.787846 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.787837    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd4c58db5-r9djk: secret "image-registry-tls" not found
Jul 13 20:58:05.787960 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.787900    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls podName:41e2a709-c033-4b04-99ef-c5876668a5b1 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:07.787880628 +0000 UTC m=+248.726084918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls") pod "image-registry-6bd4c58db5-r9djk" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1") : secret "image-registry-tls" not found
Jul 13 20:58:05.888676 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.888639    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:05.888868 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.888718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:05.888868 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.888783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:05.888868 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.888822    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 20:58:05.888868 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:05.888833    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:05.889082 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.888896    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls podName:b9dfd806-8ff8-4ff5-b845-ae9eba86ab64 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:07.888876263 +0000 UTC m=+248.827080557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls") pod "cluster-samples-operator-65db858894-th7z9" (UID: "b9dfd806-8ff8-4ff5-b845-ae9eba86ab64") : secret "samples-operator-tls" not found
Jul 13 20:58:05.889082 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.888947    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:05.889082 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.888953    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 20:58:05.889082 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.888986    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:07.888963718 +0000 UTC m=+248.827168017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : configmap references non-existent config key: service-ca.crt
Jul 13 20:58:05.889082 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.889018    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:07.889005716 +0000 UTC m=+248.827210007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : secret "router-metrics-certs-default" not found
Jul 13 20:58:05.889082 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:05.889036    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls podName:90d21d66-81d6-464c-a857-7d43f52beb1d nodeName:}" failed. No retries permitted until 2026-07-13 20:58:07.8890273 +0000 UTC m=+248.827231593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2vxxv" (UID: "90d21d66-81d6-464c-a857-7d43f52beb1d") : secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:07.805164 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:07.805136    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:07.805489 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.805286    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 20:58:07.805489 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.805304    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd4c58db5-r9djk: secret "image-registry-tls" not found
Jul 13 20:58:07.805489 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.805383    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls podName:41e2a709-c033-4b04-99ef-c5876668a5b1 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:11.805363197 +0000 UTC m=+252.743567488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls") pod "image-registry-6bd4c58db5-r9djk" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1") : secret "image-registry-tls" not found
Jul 13 20:58:07.905650 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:07.905627    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:07.905742 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:07.905671    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:07.905742 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:07.905716    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:07.905742 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:07.905741    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:07.905879 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.905811    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 20:58:07.905928 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.905878    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 20:58:07.905928 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.905884    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:11.905863521 +0000 UTC m=+252.844067816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : secret "router-metrics-certs-default" not found
Jul 13 20:58:07.906035 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.905939    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:11.905922263 +0000 UTC m=+252.844126558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : configmap references non-existent config key: service-ca.crt
Jul 13 20:58:07.906035 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.905960    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls podName:b9dfd806-8ff8-4ff5-b845-ae9eba86ab64 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:11.905950919 +0000 UTC m=+252.844155212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls") pod "cluster-samples-operator-65db858894-th7z9" (UID: "b9dfd806-8ff8-4ff5-b845-ae9eba86ab64") : secret "samples-operator-tls" not found
Jul 13 20:58:07.906035 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.905976    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:07.906035 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:07.906031    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls podName:90d21d66-81d6-464c-a857-7d43f52beb1d nodeName:}" failed. No retries permitted until 2026-07-13 20:58:11.906015557 +0000 UTC m=+252.844219852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2vxxv" (UID: "90d21d66-81d6-464c-a857-7d43f52beb1d") : secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:08.305806 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.305776    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/0.log"
Jul 13 20:58:08.305985 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.305813    2556 generic.go:356] "Generic (PLEG): container finished" podID="0cd5c58a-b847-450e-aebd-3a21b97e4f10" containerID="5c6e271283b95ee2dd24244cac490af69e3a35bfb0e72502738f72de5404b04d" exitCode=255
Jul 13 20:58:08.305985 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.305875    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gd4wz" event={"ID":"0cd5c58a-b847-450e-aebd-3a21b97e4f10","Type":"ContainerDied","Data":"5c6e271283b95ee2dd24244cac490af69e3a35bfb0e72502738f72de5404b04d"}
Jul 13 20:58:08.306162 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.306145    2556 scope.go:117] "RemoveContainer" containerID="5c6e271283b95ee2dd24244cac490af69e3a35bfb0e72502738f72de5404b04d"
Jul 13 20:58:08.307533 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.307507    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9" event={"ID":"f7e89441-6447-4b44-b1e3-b3a8ead9aa13","Type":"ContainerStarted","Data":"b64f34ae09b5bb9ad1ece26b8594fb76a3637ccb049372d9c89f7583c3ecd67b"}
Jul 13 20:58:08.309026 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.308980    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z" event={"ID":"7518b6cb-fdc2-46b6-934c-e73079d7a0d2","Type":"ContainerStarted","Data":"905c94624f6980a26a0e0b805acd92caedb80bbf86a5d2d55a7df28a1d03e66c"}
Jul 13 20:58:08.310571 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.310554    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc" event={"ID":"97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2","Type":"ContainerStarted","Data":"30309f4549ec16402978d4600904011f216cd8e24accf2fcc4665bb74dfe94f6"}
Jul 13 20:58:08.338452 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.338403    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z" podStartSLOduration=1.195288539 podStartE2EDuration="4.338386039s" totalImagesPullingTime="3.1430975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:08.337098118 +0000 UTC m=+249.275302433" watchObservedRunningTime="2026-07-13 20:58:08.338386039 +0000 UTC m=+249.276590352"
Jul 13 20:58:08.352431 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.352383    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqddc" podStartSLOduration=1.555822493 podStartE2EDuration="4.352363228s" totalImagesPullingTime="2.796540735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:08.351062269 +0000 UTC m=+249.289266592" watchObservedRunningTime="2026-07-13 20:58:08.352363228 +0000 UTC m=+249.290567541"
Jul 13 20:58:08.369123 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:08.368441    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9" podStartSLOduration=1.230341204 podStartE2EDuration="4.368421596s" totalImagesPullingTime="3.138080392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:08.367383436 +0000 UTC m=+249.305587746" watchObservedRunningTime="2026-07-13 20:58:08.368421596 +0000 UTC m=+249.306625908"
Jul 13 20:58:09.134877 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.134839    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w8tqk"]
Jul 13 20:58:09.137159 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.137138    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.145374 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.141938    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pgh4v\""
Jul 13 20:58:09.146182 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.146128    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 20:58:09.146182 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.146175    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 20:58:09.153154 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.153129    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w8tqk"]
Jul 13 20:58:09.217909 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.217876    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2c8z\" (UniqueName: \"kubernetes.io/projected/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-api-access-s2c8z\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.218047 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.217922    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.218047 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.218005    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee9564f-ce55-439b-b58c-89c8721bd217-data-volume\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.218047 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.218044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ee9564f-ce55-439b-b58c-89c8721bd217-crio-socket\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.218186 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.218061    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.315303 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.315277    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 20:58:09.315711 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.315695    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/0.log"
Jul 13 20:58:09.315784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.315727    2556 generic.go:356] "Generic (PLEG): container finished" podID="0cd5c58a-b847-450e-aebd-3a21b97e4f10" containerID="0cad8ea6e57a2097d2bac797cf9d197e8297e0791a7a77f20f03d976a89bb7ad" exitCode=255
Jul 13 20:58:09.315824 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.315814    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gd4wz" event={"ID":"0cd5c58a-b847-450e-aebd-3a21b97e4f10","Type":"ContainerDied","Data":"0cad8ea6e57a2097d2bac797cf9d197e8297e0791a7a77f20f03d976a89bb7ad"}
Jul 13 20:58:09.315866 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.315851    2556 scope.go:117] "RemoveContainer" containerID="5c6e271283b95ee2dd24244cac490af69e3a35bfb0e72502738f72de5404b04d"
Jul 13 20:58:09.316110 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.316085    2556 scope.go:117] "RemoveContainer" containerID="0cad8ea6e57a2097d2bac797cf9d197e8297e0791a7a77f20f03d976a89bb7ad"
Jul 13 20:58:09.316322 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.316296    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gd4wz_openshift-console-operator(0cd5c58a-b847-450e-aebd-3a21b97e4f10)\"" pod="openshift-console-operator/console-operator-7866497494-gd4wz" podUID="0cd5c58a-b847-450e-aebd-3a21b97e4f10"
Jul 13 20:58:09.318903 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.318885    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2c8z\" (UniqueName: \"kubernetes.io/projected/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-api-access-s2c8z\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.318990 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.318920    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.318990 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.318960    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee9564f-ce55-439b-b58c-89c8721bd217-data-volume\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.319080 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.319011    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ee9564f-ce55-439b-b58c-89c8721bd217-crio-socket\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.319080 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.319034    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.319165 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.319036    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:09.319165 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.319094    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ee9564f-ce55-439b-b58c-89c8721bd217-crio-socket\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.319165 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.319105    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 20:58:09.319165 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.319139    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:09.81912167 +0000 UTC m=+250.757325967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:09.319165 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.319154    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:09.819147109 +0000 UTC m=+250.757351398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : configmap "kube-rbac-proxy" not found
Jul 13 20:58:09.319396 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.319375    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ee9564f-ce55-439b-b58c-89c8721bd217-data-volume\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.326319 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.326294    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2c8z\" (UniqueName: \"kubernetes.io/projected/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-api-access-s2c8z\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.823845 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.823810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.824116 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.823929    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 20:58:09.824116 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:09.823967    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:09.824116 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.823991    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:10.823975494 +0000 UTC m=+251.762179786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : configmap "kube-rbac-proxy" not found
Jul 13 20:58:09.824116 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.824047    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:09.824116 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:09.824105    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:10.824091167 +0000 UTC m=+251.762295461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:10.319880 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:10.319848    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 20:58:10.320369 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:10.320261    2556 scope.go:117] "RemoveContainer" containerID="0cad8ea6e57a2097d2bac797cf9d197e8297e0791a7a77f20f03d976a89bb7ad"
Jul 13 20:58:10.320488 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:10.320466    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gd4wz_openshift-console-operator(0cd5c58a-b847-450e-aebd-3a21b97e4f10)\"" pod="openshift-console-operator/console-operator-7866497494-gd4wz" podUID="0cd5c58a-b847-450e-aebd-3a21b97e4f10"
Jul 13 20:58:10.832660 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:10.832625    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:10.832884 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:10.832704    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:10.832884 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:10.832779    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:10.832884 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:10.832801    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 20:58:10.832884 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:10.832851    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:12.832836869 +0000 UTC m=+253.771041158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : configmap "kube-rbac-proxy" not found
Jul 13 20:58:10.832884 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:10.832864    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:12.832858725 +0000 UTC m=+253.771063013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:11.335643 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:11.335612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:58:11.336035 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.335754    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 13 20:58:11.336035 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.335813    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs podName:de4035ca-3984-435e-8387-217677bcc954 nodeName:}" failed. No retries permitted until 2026-07-13 21:00:13.335797934 +0000 UTC m=+374.274002223 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs") pod "network-metrics-daemon-grcrp" (UID: "de4035ca-3984-435e-8387-217677bcc954") : secret "metrics-daemon-secret" not found
Jul 13 20:58:11.729143 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:11.729120    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n688c_38b9c8cd-9d66-430c-b411-e92d0a730deb/dns-node-resolver/0.log"
Jul 13 20:58:11.839509 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:11.839469    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:11.839681 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.839617    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 20:58:11.839681 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.839637    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd4c58db5-r9djk: secret "image-registry-tls" not found
Jul 13 20:58:11.839750 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.839696    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls podName:41e2a709-c033-4b04-99ef-c5876668a5b1 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:19.83967857 +0000 UTC m=+260.777882863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls") pod "image-registry-6bd4c58db5-r9djk" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1") : secret "image-registry-tls" not found
Jul 13 20:58:11.940980 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:11.940948    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:11.941173 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:11.941004    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:11.941173 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:11.941051    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:11.941173 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.941124    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 20:58:11.941173 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.941127    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 20:58:11.941173 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.941155    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:11.941404 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.941181    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls podName:b9dfd806-8ff8-4ff5-b845-ae9eba86ab64 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:19.941164377 +0000 UTC m=+260.879368690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls") pod "cluster-samples-operator-65db858894-th7z9" (UID: "b9dfd806-8ff8-4ff5-b845-ae9eba86ab64") : secret "samples-operator-tls" not found
Jul 13 20:58:11.941404 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.941197    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls podName:90d21d66-81d6-464c-a857-7d43f52beb1d nodeName:}" failed. No retries permitted until 2026-07-13 20:58:19.941191307 +0000 UTC m=+260.879395596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-2vxxv" (UID: "90d21d66-81d6-464c-a857-7d43f52beb1d") : secret "cluster-monitoring-operator-tls" not found
Jul 13 20:58:11.941404 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.941217    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:19.94121155 +0000 UTC m=+260.879415839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : secret "router-metrics-certs-default" not found
Jul 13 20:58:11.941404 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:11.941253    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:11.941404 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:11.941379    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle podName:f0172f3d-38f3-4259-88f8-258f6424f8e0 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:19.941369141 +0000 UTC m=+260.879573433 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle") pod "router-default-7488d7cbb8-nn48c" (UID: "f0172f3d-38f3-4259-88f8-258f6424f8e0") : configmap references non-existent config key: service-ca.crt
Jul 13 20:58:12.848194 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:12.848159    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:12.848681 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:12.848241    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:12.848681 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:12.848299    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 20:58:12.848681 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:12.848327    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:12.848681 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:12.848400    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:16.848379122 +0000 UTC m=+257.786583421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : configmap "kube-rbac-proxy" not found
Jul 13 20:58:12.848681 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:12.848422    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:16.848411792 +0000 UTC m=+257.786616080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:13.127659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:13.127634    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9pjnw_3ba71b3d-1630-4cd4-912c-2ed8f84aad46/node-ca/0.log"
Jul 13 20:58:13.527701 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:13.527631    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7gknj_20d69c93-0f84-46ed-95ab-2d19b1515e04/serve-healthcheck-canary/0.log"
Jul 13 20:58:14.534255 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:14.534222    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bkql9_f7e89441-6447-4b44-b1e3-b3a8ead9aa13/kube-storage-version-migrator-operator/0.log"
Jul 13 20:58:14.568445 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:14.568417    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:14.568581 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:14.568465    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:14.568771 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:14.568759    2556 scope.go:117] "RemoveContainer" containerID="0cad8ea6e57a2097d2bac797cf9d197e8297e0791a7a77f20f03d976a89bb7ad"
Jul 13 20:58:14.568908 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:14.568891    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gd4wz_openshift-console-operator(0cd5c58a-b847-450e-aebd-3a21b97e4f10)\"" pod="openshift-console-operator/console-operator-7866497494-gd4wz" podUID="0cd5c58a-b847-450e-aebd-3a21b97e4f10"
Jul 13 20:58:16.878250 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:16.878214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:16.878690 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:16.878310    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:16.878690 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:16.878374    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:16.878690 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:16.878425    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 20:58:16.878690 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:16.878435    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:24.878419049 +0000 UTC m=+265.816623339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : secret "insights-runtime-extractor-tls" not found
Jul 13 20:58:16.878690 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:16.878484    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm podName:6ee9564f-ce55-439b-b58c-89c8721bd217 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:24.878468891 +0000 UTC m=+265.816673183 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8tqk" (UID: "6ee9564f-ce55-439b-b58c-89c8721bd217") : configmap "kube-rbac-proxy" not found
Jul 13 20:58:19.902546 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:19.902514    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:19.904800 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:19.904782    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"image-registry-6bd4c58db5-r9djk\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") " pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:20.003845 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.003813    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:20.004009 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.003861    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:20.004009 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.003905    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:20.004009 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.003954    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:20.004590 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.004566    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0172f3d-38f3-4259-88f8-258f6424f8e0-service-ca-bundle\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:20.006302 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.006280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90d21d66-81d6-464c-a857-7d43f52beb1d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-2vxxv\" (UID: \"90d21d66-81d6-464c-a857-7d43f52beb1d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:20.006379 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.006360    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9dfd806-8ff8-4ff5-b845-ae9eba86ab64-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-th7z9\" (UID: \"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:20.006464 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.006449    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0172f3d-38f3-4259-88f8-258f6424f8e0-metrics-certs\") pod \"router-default-7488d7cbb8-nn48c\" (UID: \"f0172f3d-38f3-4259-88f8-258f6424f8e0\") " pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:20.074833 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.074813    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:20.080994 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.080980    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:20.084782 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.084762    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"
Jul 13 20:58:20.173774 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.173745    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"
Jul 13 20:58:20.222253 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.222219    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bd4c58db5-r9djk"]
Jul 13 20:58:20.225730 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:20.225704    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e2a709_c033_4b04_99ef_c5876668a5b1.slice/crio-e75d0424ab69635a8e645784744e262973038ea49873e54a54feec9169df5ea8 WatchSource:0}: Error finding container e75d0424ab69635a8e645784744e262973038ea49873e54a54feec9169df5ea8: Status 404 returned error can't find the container with id e75d0424ab69635a8e645784744e262973038ea49873e54a54feec9169df5ea8
Jul 13 20:58:20.311406 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.311379    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9"]
Jul 13 20:58:20.351473 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.351444    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" event={"ID":"41e2a709-c033-4b04-99ef-c5876668a5b1","Type":"ContainerStarted","Data":"ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f"}
Jul 13 20:58:20.351582 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.351499    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" event={"ID":"41e2a709-c033-4b04-99ef-c5876668a5b1","Type":"ContainerStarted","Data":"e75d0424ab69635a8e645784744e262973038ea49873e54a54feec9169df5ea8"}
Jul 13 20:58:20.351651 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.351637    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:20.373781 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.373738    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" podStartSLOduration=16.373723482 podStartE2EDuration="16.373723482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:20.37278101 +0000 UTC m=+261.310985347" watchObservedRunningTime="2026-07-13 20:58:20.373723482 +0000 UTC m=+261.311927793"
Jul 13 20:58:20.442628 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.442577    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv"]
Jul 13 20:58:20.445775 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:20.445742    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d21d66_81d6_464c_a857_7d43f52beb1d.slice/crio-ad782d850b090276c75bd069f0336308204554cbe12c47eaabcd602aa374620c WatchSource:0}: Error finding container ad782d850b090276c75bd069f0336308204554cbe12c47eaabcd602aa374620c: Status 404 returned error can't find the container with id ad782d850b090276c75bd069f0336308204554cbe12c47eaabcd602aa374620c
Jul 13 20:58:20.446232 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:20.446211    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7488d7cbb8-nn48c"]
Jul 13 20:58:20.448368 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:20.448304    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0172f3d_38f3_4259_88f8_258f6424f8e0.slice/crio-bd2a48eb795be88f3dada34217bac70c4301e68687ebd45320b8c6da1a36d486 WatchSource:0}: Error finding container bd2a48eb795be88f3dada34217bac70c4301e68687ebd45320b8c6da1a36d486: Status 404 returned error can't find the container with id bd2a48eb795be88f3dada34217bac70c4301e68687ebd45320b8c6da1a36d486
Jul 13 20:58:21.357368 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:21.357258    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9" event={"ID":"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64","Type":"ContainerStarted","Data":"5204a10651b0c473ce1c5d300bcc097e39bece249e8aba738c334605563385d9"}
Jul 13 20:58:21.359156 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:21.359127    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7488d7cbb8-nn48c" event={"ID":"f0172f3d-38f3-4259-88f8-258f6424f8e0","Type":"ContainerStarted","Data":"8d8b103900908d0cae23c68edef2cd6e83a487db1260e8748d84daf29ccc7019"}
Jul 13 20:58:21.359272 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:21.359175    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7488d7cbb8-nn48c" event={"ID":"f0172f3d-38f3-4259-88f8-258f6424f8e0","Type":"ContainerStarted","Data":"bd2a48eb795be88f3dada34217bac70c4301e68687ebd45320b8c6da1a36d486"}
Jul 13 20:58:21.360282 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:21.360255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv" event={"ID":"90d21d66-81d6-464c-a857-7d43f52beb1d","Type":"ContainerStarted","Data":"ad782d850b090276c75bd069f0336308204554cbe12c47eaabcd602aa374620c"}
Jul 13 20:58:21.380059 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:21.380017    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7488d7cbb8-nn48c" podStartSLOduration=17.380004166 podStartE2EDuration="17.380004166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:21.378180498 +0000 UTC m=+262.316384821" watchObservedRunningTime="2026-07-13 20:58:21.380004166 +0000 UTC m=+262.318208477"
Jul 13 20:58:22.081912 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:22.081877    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:22.085159 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:22.085134    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:22.365153 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:22.365126    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:22.366468 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:22.366447    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7488d7cbb8-nn48c"
Jul 13 20:58:23.374108 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:23.374072    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9" event={"ID":"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64","Type":"ContainerStarted","Data":"f295df8590cf2f177a9327acf5fb6da589b06abc0000c03e6986487a4f598fb4"}
Jul 13 20:58:23.374545 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:23.374158    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9" event={"ID":"b9dfd806-8ff8-4ff5-b845-ae9eba86ab64","Type":"ContainerStarted","Data":"0a393eeb941b85430ffcc3285d63a90b881cb87ebf617d82951545b7ac729b28"}
Jul 13 20:58:23.375468 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:23.375441    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv" event={"ID":"90d21d66-81d6-464c-a857-7d43f52beb1d","Type":"ContainerStarted","Data":"1a297b36f1cf39d1c7e7ec6067e12d330d072233cdbf5ca603becd6a5597cff7"}
Jul 13 20:58:23.390853 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:23.390815    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-th7z9" podStartSLOduration=16.788914722 podStartE2EDuration="19.390802539s" totalImagesPullingTime="2.601887817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:23.390249655 +0000 UTC m=+264.328453964" watchObservedRunningTime="2026-07-13 20:58:23.390802539 +0000 UTC m=+264.329006849"
Jul 13 20:58:23.406804 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:23.406758    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-2vxxv" podStartSLOduration=16.896318636 podStartE2EDuration="19.406742893s" totalImagesPullingTime="2.510424257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:23.406654331 +0000 UTC m=+264.344858668" watchObservedRunningTime="2026-07-13 20:58:23.406742893 +0000 UTC m=+264.344947205"
Jul 13 20:58:24.947541 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:24.947509    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:24.947889 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:24.947576    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:24.948020 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:24.947999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ee9564f-ce55-439b-b58c-89c8721bd217-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:24.949760 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:24.949744    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ee9564f-ce55-439b-b58c-89c8721bd217-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8tqk\" (UID: \"6ee9564f-ce55-439b-b58c-89c8721bd217\") " pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:25.051589 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:25.051564    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w8tqk"
Jul 13 20:58:25.173968 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:25.173946    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w8tqk"]
Jul 13 20:58:25.176405 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:25.176379    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee9564f_ce55_439b_b58c_89c8721bd217.slice/crio-4f5060af13773f3ff725b294a425c02ab64c78220a698e3906669168919deb50 WatchSource:0}: Error finding container 4f5060af13773f3ff725b294a425c02ab64c78220a698e3906669168919deb50: Status 404 returned error can't find the container with id 4f5060af13773f3ff725b294a425c02ab64c78220a698e3906669168919deb50
Jul 13 20:58:25.382576 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:25.382542    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8tqk" event={"ID":"6ee9564f-ce55-439b-b58c-89c8721bd217","Type":"ContainerStarted","Data":"4f5060af13773f3ff725b294a425c02ab64c78220a698e3906669168919deb50"}
Jul 13 20:58:29.398616 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:29.398589    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8tqk" event={"ID":"6ee9564f-ce55-439b-b58c-89c8721bd217","Type":"ContainerStarted","Data":"3bb1c9f98ea545c8ef47a2248434decccc49c9c2b58d75b007ce0c6a773db90e"}
Jul 13 20:58:29.601526 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:29.601440    2556 scope.go:117] "RemoveContainer" containerID="0cad8ea6e57a2097d2bac797cf9d197e8297e0791a7a77f20f03d976a89bb7ad"
Jul 13 20:58:30.403841 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:30.403813    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 20:58:30.404298 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:30.403943    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gd4wz" event={"ID":"0cd5c58a-b847-450e-aebd-3a21b97e4f10","Type":"ContainerStarted","Data":"d483de2ccd6e41480cd51f57e6a939395f6407d23581a342f1c758c560cdc31d"}
Jul 13 20:58:30.404298 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:30.404273    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:30.405870 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:30.405844    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8tqk" event={"ID":"6ee9564f-ce55-439b-b58c-89c8721bd217","Type":"ContainerStarted","Data":"dac6b338e69ef31111e85775460c3e5edac162ee22d2fbd06e0cbec408685f3e"}
Jul 13 20:58:30.410011 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:30.409987    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-gd4wz"
Jul 13 20:58:30.423398 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:30.423359    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-gd4wz" podStartSLOduration=23.602568233 podStartE2EDuration="26.423322686s" totalImagesPullingTime="2.820754453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:30.422221179 +0000 UTC m=+271.360425490" watchObservedRunningTime="2026-07-13 20:58:30.423322686 +0000 UTC m=+271.361527058"
Jul 13 20:58:32.415221 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:32.415181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8tqk" event={"ID":"6ee9564f-ce55-439b-b58c-89c8721bd217","Type":"ContainerStarted","Data":"c94bc229686207e5f35a3c45f6a9c2e87585d90f592a8950014d9d928920dd1b"}
Jul 13 20:58:32.432105 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:32.432064    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w8tqk" podStartSLOduration=16.866295271 podStartE2EDuration="23.432049227s" totalImagesPullingTime="6.565753956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:58:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:32.431597733 +0000 UTC m=+273.369802047" watchObservedRunningTime="2026-07-13 20:58:32.432049227 +0000 UTC m=+273.370253539"
Jul 13 20:58:36.437540 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:36.437511    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bd4c58db5-r9djk"]
Jul 13 20:58:36.441963 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:36.441936    2556 patch_prober.go:28] interesting pod/image-registry-6bd4c58db5-r9djk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 13 20:58:36.442092 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:36.441979    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" podUID="41e2a709-c033-4b04-99ef-c5876668a5b1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 20:58:38.720033 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.720002    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c54bf59b6-r2gnr"]
Jul 13 20:58:38.722883 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.722864    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.725756 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.725733    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 20:58:38.725863 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.725812    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 13 20:58:38.725923 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.725743    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 13 20:58:38.726881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.726862    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 20:58:38.726985 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.726870    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 13 20:58:38.726985 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.726969    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 13 20:58:38.727119 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.727059    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-r8vlv\""
Jul 13 20:58:38.727164 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.727136    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 13 20:58:38.735503 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.735483    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c54bf59b6-r2gnr"]
Jul 13 20:58:38.856534 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.856506    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76lp8\" (UniqueName: \"kubernetes.io/projected/6dfedac0-64d5-4664-ac18-755e5239120a-kube-api-access-76lp8\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.856694 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.856542    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-service-ca\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.856694 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.856571    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-oauth-serving-cert\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.856694 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.856595    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-serving-cert\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.856694 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.856616    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-console-config\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.856694 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.856669    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-oauth-config\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.957809 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.957776    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-oauth-config\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.957967 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.957815    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76lp8\" (UniqueName: \"kubernetes.io/projected/6dfedac0-64d5-4664-ac18-755e5239120a-kube-api-access-76lp8\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.957967 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.957847    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-service-ca\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.957967 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.957884    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-oauth-serving-cert\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.957967 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.957907    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-serving-cert\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.957967 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.957929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-console-config\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.958625 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.958602    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-oauth-serving-cert\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.958625 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.958617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-service-ca\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.958803 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.958772    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-console-config\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.960236 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.960213    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-oauth-config\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.960329 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.960319    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-serving-cert\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:38.966488 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:38.966464    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76lp8\" (UniqueName: \"kubernetes.io/projected/6dfedac0-64d5-4664-ac18-755e5239120a-kube-api-access-76lp8\") pod \"console-6c54bf59b6-r2gnr\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") " pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:39.031635 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.031558    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:39.160448 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.160422    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c54bf59b6-r2gnr"]
Jul 13 20:58:39.162234 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:39.162196    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dfedac0_64d5_4664_ac18_755e5239120a.slice/crio-c5dde5c4ff383c205e005d4d5fc0fc22bc60650a76c0fafede8a16589647a06b WatchSource:0}: Error finding container c5dde5c4ff383c205e005d4d5fc0fc22bc60650a76c0fafede8a16589647a06b: Status 404 returned error can't find the container with id c5dde5c4ff383c205e005d4d5fc0fc22bc60650a76c0fafede8a16589647a06b
Jul 13 20:58:39.437798 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.437766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c54bf59b6-r2gnr" event={"ID":"6dfedac0-64d5-4664-ac18-755e5239120a","Type":"ContainerStarted","Data":"c5dde5c4ff383c205e005d4d5fc0fc22bc60650a76c0fafede8a16589647a06b"}
Jul 13 20:58:39.520535 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.520495    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vqpcz"]
Jul 13 20:58:39.524443 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.524422    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.526773 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.526752    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fmfws\""
Jul 13 20:58:39.527792 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.527772    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 20:58:39.527900 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.527806    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 13 20:58:39.527900 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.527829    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 13 20:58:39.531943 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.531921    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vqpcz"]
Jul 13 20:58:39.662254 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.662223    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/89757365-9849-4ec4-b600-5480a74b265e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.662446 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.662269    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89757365-9849-4ec4-b600-5480a74b265e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.662446 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.662301    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8msr\" (UniqueName: \"kubernetes.io/projected/89757365-9849-4ec4-b600-5480a74b265e-kube-api-access-p8msr\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.662446 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.662386    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89757365-9849-4ec4-b600-5480a74b265e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.763109 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.763021    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89757365-9849-4ec4-b600-5480a74b265e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.763109 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.763069    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8msr\" (UniqueName: \"kubernetes.io/projected/89757365-9849-4ec4-b600-5480a74b265e-kube-api-access-p8msr\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.763629 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.763128    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89757365-9849-4ec4-b600-5480a74b265e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.763629 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.763174    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/89757365-9849-4ec4-b600-5480a74b265e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.763898 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.763874    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89757365-9849-4ec4-b600-5480a74b265e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.765574 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.765550    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89757365-9849-4ec4-b600-5480a74b265e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.765710 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.765694    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/89757365-9849-4ec4-b600-5480a74b265e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.771189 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.771169    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8msr\" (UniqueName: \"kubernetes.io/projected/89757365-9849-4ec4-b600-5480a74b265e-kube-api-access-p8msr\") pod \"prometheus-operator-64785b6675-vqpcz\" (UID: \"89757365-9849-4ec4-b600-5480a74b265e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.835008 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.834978    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz"
Jul 13 20:58:39.965572 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:39.965544    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vqpcz"]
Jul 13 20:58:39.968127 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:39.968099    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89757365_9849_4ec4_b600_5480a74b265e.slice/crio-30509af93886b133001feeaefb119166720f5459375b3f71996081e8be7d6a74 WatchSource:0}: Error finding container 30509af93886b133001feeaefb119166720f5459375b3f71996081e8be7d6a74: Status 404 returned error can't find the container with id 30509af93886b133001feeaefb119166720f5459375b3f71996081e8be7d6a74
Jul 13 20:58:40.442545 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:40.442511    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz" event={"ID":"89757365-9849-4ec4-b600-5480a74b265e","Type":"ContainerStarted","Data":"30509af93886b133001feeaefb119166720f5459375b3f71996081e8be7d6a74"}
Jul 13 20:58:41.087654 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:41.087573    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4fxfq" podUID="326c721f-7a82-4669-b58c-153c8ac68f50"
Jul 13 20:58:41.445775 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:41.445742    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:58:43.454798 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:43.454762    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c54bf59b6-r2gnr" event={"ID":"6dfedac0-64d5-4664-ac18-755e5239120a","Type":"ContainerStarted","Data":"772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd"}
Jul 13 20:58:43.456453 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:43.456424    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz" event={"ID":"89757365-9849-4ec4-b600-5480a74b265e","Type":"ContainerStarted","Data":"4639a414e6e319a2b12c5da147b1813e43b489acb58bdf89095a080525f9cf31"}
Jul 13 20:58:43.456570 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:43.456470    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz" event={"ID":"89757365-9849-4ec4-b600-5480a74b265e","Type":"ContainerStarted","Data":"f68b39b6e902cec6790c13acc0e01b5656b8eda474127139b522cba1bbf7e924"}
Jul 13 20:58:43.477105 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:43.477063    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c54bf59b6-r2gnr" podStartSLOduration=1.678202145 podStartE2EDuration="5.477050325s" totalImagesPullingTime="3.79884818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:43.476387942 +0000 UTC m=+284.414592252" watchObservedRunningTime="2026-07-13 20:58:43.477050325 +0000 UTC m=+284.415254636"
Jul 13 20:58:43.493064 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:43.492948    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-vqpcz" podStartSLOduration=1.50623929 podStartE2EDuration="4.492933625s" totalImagesPullingTime="2.986694335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:43.49282942 +0000 UTC m=+284.431033732" watchObservedRunningTime="2026-07-13 20:58:43.492933625 +0000 UTC m=+284.431137937"
Jul 13 20:58:44.804958 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:44.804918    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:58:44.807194 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:44.807174    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/326c721f-7a82-4669-b58c-153c8ac68f50-metrics-tls\") pod \"dns-default-4fxfq\" (UID: \"326c721f-7a82-4669-b58c-153c8ac68f50\") " pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:58:45.049148 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.049120    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f67kb\""
Jul 13 20:58:45.057759 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.057692    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:58:45.220114 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.220085    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4fxfq"]
Jul 13 20:58:45.222920 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:45.222896    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod326c721f_7a82_4669_b58c_153c8ac68f50.slice/crio-a0858b6d9eac4ea5d4b1e878768456c70fbbd03b40a395b903622679333cad29 WatchSource:0}: Error finding container a0858b6d9eac4ea5d4b1e878768456c70fbbd03b40a395b903622679333cad29: Status 404 returned error can't find the container with id a0858b6d9eac4ea5d4b1e878768456c70fbbd03b40a395b903622679333cad29
Jul 13 20:58:45.464571 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.464542    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4fxfq" event={"ID":"326c721f-7a82-4669-b58c-153c8ac68f50","Type":"ContainerStarted","Data":"a0858b6d9eac4ea5d4b1e878768456c70fbbd03b40a395b903622679333cad29"}
Jul 13 20:58:45.906634 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.906597    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"]
Jul 13 20:58:45.928145 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.928067    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wgzt9"]
Jul 13 20:58:45.928306 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.928161    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:45.931333 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.931307    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 13 20:58:45.931615 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.931596    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 13 20:58:45.932018 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.931983    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-d7b6p\""
Jul 13 20:58:45.943359 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.943324    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"]
Jul 13 20:58:45.943488 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.943471    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:45.946498 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.946479    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 20:58:45.946752 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.946733    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 20:58:45.947004 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.946985    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gxcf2\""
Jul 13 20:58:45.947086 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:45.946989    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 20:58:46.115672 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115640    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-root\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.115672 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115673    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwrfk\" (UniqueName: \"kubernetes.io/projected/acefebc3-f576-4baa-9913-d83d1e381678-kube-api-access-rwrfk\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.115881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115705    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-wtmp\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.115881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115753    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acefebc3-f576-4baa-9913-d83d1e381678-metrics-client-ca\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.115881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115780    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f1c8eac-1876-4340-8b83-feccc327e79a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.115881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115800    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.115881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115817    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-textfile\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.116106 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115922    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.116106 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.115963    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f1c8eac-1876-4340-8b83-feccc327e79a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.116106 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.116002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq4mw\" (UniqueName: \"kubernetes.io/projected/9f1c8eac-1876-4340-8b83-feccc327e79a-kube-api-access-pq4mw\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.116106 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.116039    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-sys\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.116106 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.116067    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-tls\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.116106 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.116106    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f1c8eac-1876-4340-8b83-feccc327e79a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.216843 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.216766    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-wtmp\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.217016 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.216853    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acefebc3-f576-4baa-9913-d83d1e381678-metrics-client-ca\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.217016 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.216888    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f1c8eac-1876-4340-8b83-feccc327e79a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.217016 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.216914    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.217016 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.216940    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-textfile\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.217016 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217028    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f1c8eac-1876-4340-8b83-feccc327e79a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217056    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4mw\" (UniqueName: \"kubernetes.io/projected/9f1c8eac-1876-4340-8b83-feccc327e79a-kube-api-access-pq4mw\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217079    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-sys\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217102    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-tls\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217102    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-wtmp\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217136    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f1c8eac-1876-4340-8b83-feccc327e79a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217171    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-root\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217195    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwrfk\" (UniqueName: \"kubernetes.io/projected/acefebc3-f576-4baa-9913-d83d1e381678-kube-api-access-rwrfk\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217366    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-root\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217784    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acefebc3-f576-4baa-9913-d83d1e381678-metrics-client-ca\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217885    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-textfile\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217884    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.217952    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acefebc3-f576-4baa-9913-d83d1e381678-sys\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.218220 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.218093    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f1c8eac-1876-4340-8b83-feccc327e79a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.220068 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.220037    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f1c8eac-1876-4340-8b83-feccc327e79a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.220252 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.220230    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f1c8eac-1876-4340-8b83-feccc327e79a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.220326 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.220257    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-tls\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.220518 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.220500    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acefebc3-f576-4baa-9913-d83d1e381678-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.225633 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.225605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwrfk\" (UniqueName: \"kubernetes.io/projected/acefebc3-f576-4baa-9913-d83d1e381678-kube-api-access-rwrfk\") pod \"node-exporter-wgzt9\" (UID: \"acefebc3-f576-4baa-9913-d83d1e381678\") " pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.226085 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.226067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq4mw\" (UniqueName: \"kubernetes.io/projected/9f1c8eac-1876-4340-8b83-feccc327e79a-kube-api-access-pq4mw\") pod \"openshift-state-metrics-84bd68b896-c97lj\" (UID: \"9f1c8eac-1876-4340-8b83-feccc327e79a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.241151 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.241129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"
Jul 13 20:58:46.253309 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.253282    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wgzt9"
Jul 13 20:58:46.308702 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:46.308666    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacefebc3_f576_4baa_9913_d83d1e381678.slice/crio-76c5d8a912d2d893de560ceb1b246bd677580f9b0c8d635c87fd94bf00fa8c99 WatchSource:0}: Error finding container 76c5d8a912d2d893de560ceb1b246bd677580f9b0c8d635c87fd94bf00fa8c99: Status 404 returned error can't find the container with id 76c5d8a912d2d893de560ceb1b246bd677580f9b0c8d635c87fd94bf00fa8c99
Jul 13 20:58:46.417364 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.417311    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj"]
Jul 13 20:58:46.420649 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:46.420621    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1c8eac_1876_4340_8b83_feccc327e79a.slice/crio-d73bf284f98508d314bfe347558975c351537b7cf9722cc926136572d9ffca72 WatchSource:0}: Error finding container d73bf284f98508d314bfe347558975c351537b7cf9722cc926136572d9ffca72: Status 404 returned error can't find the container with id d73bf284f98508d314bfe347558975c351537b7cf9722cc926136572d9ffca72
Jul 13 20:58:46.442554 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.442523    2556 patch_prober.go:28] interesting pod/image-registry-6bd4c58db5-r9djk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 13 20:58:46.442648 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.442561    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" podUID="41e2a709-c033-4b04-99ef-c5876668a5b1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 20:58:46.469279 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.469227    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgzt9" event={"ID":"acefebc3-f576-4baa-9913-d83d1e381678","Type":"ContainerStarted","Data":"76c5d8a912d2d893de560ceb1b246bd677580f9b0c8d635c87fd94bf00fa8c99"}
Jul 13 20:58:46.470227 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.470208    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj" event={"ID":"9f1c8eac-1876-4340-8b83-feccc327e79a","Type":"ContainerStarted","Data":"d73bf284f98508d314bfe347558975c351537b7cf9722cc926136572d9ffca72"}
Jul 13 20:58:46.970057 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.970021    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 20:58:46.999413 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.999382    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 20:58:46.999592 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:46.999512    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.002507 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.002487    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 20:58:47.002789 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.002768    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 20:58:47.002888 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.002800    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 20:58:47.003060 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.003040    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5jh9g\""
Jul 13 20:58:47.003134 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.003084    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 20:58:47.003250 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.003234    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 20:58:47.003317 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.003283    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 20:58:47.003462 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.003041    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 20:58:47.003533 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.003495    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 20:58:47.003590 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.003446    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 20:58:47.125392 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125356    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125576 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125472    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-web-config\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125576 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125542    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125681 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125577    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-config-volume\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125681 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125594    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125681 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125620    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29vhk\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-kube-api-access-29vhk\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125836 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-config-out\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125836 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125734    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125836 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125765    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.125836 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125820    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.126032 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125845    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.126032 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125885    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.126032 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.125925    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.226988 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.226917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.226988 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.226958    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-web-config\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.226997    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227029    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-config-volume\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227050    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29vhk\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-kube-api-access-29vhk\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227110    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-config-out\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227159    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227216 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:47.227192    2556 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 13 20:58:47.227613 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:47.227268    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls podName:af706187-8a93-4845-b443-d4f4838cf282 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:47.727245285 +0000 UTC m=+288.665449581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "af706187-8a93-4845-b443-d4f4838cf282") : secret "alertmanager-main-tls" not found
Jul 13 20:58:47.227613 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227613 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227479    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227613 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227517    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.227613 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.227560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.230643 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.230070    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-web-config\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.230643 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.230114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-config-out\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.230643 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.230175    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.230643 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.230273    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.230643 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:47.230424    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle podName:af706187-8a93-4845-b443-d4f4838cf282 nodeName:}" failed. No retries permitted until 2026-07-13 20:58:47.730404829 +0000 UTC m=+288.668609121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "af706187-8a93-4845-b443-d4f4838cf282") : configmap references non-existent config key: ca-bundle.crt
Jul 13 20:58:47.230643 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.230430    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.230643 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.230585    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.231048 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.230822    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.231277 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.231256    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.231527 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.231505    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.231943 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.231921    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-config-volume\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.246138 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.246117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29vhk\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-kube-api-access-29vhk\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.480269 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.479590    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj" event={"ID":"9f1c8eac-1876-4340-8b83-feccc327e79a","Type":"ContainerStarted","Data":"4e094501f87f6b5f938f0ceff3335066f01d92e55616f708792b6fadf4c93423"}
Jul 13 20:58:47.480269 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.479661    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj" event={"ID":"9f1c8eac-1876-4340-8b83-feccc327e79a","Type":"ContainerStarted","Data":"3dbf2c41877df933f122ccb236d5fdec6cc7dbdc3eae971068ceb308f3cf5590"}
Jul 13 20:58:47.731880 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.731810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.731880 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.731882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.732728 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.732703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.734454 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.734433    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.912655 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.912623    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 20:58:47.983305 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:47.983245    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5499b7b54d-qgj47"]
Jul 13 20:58:48.000187 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.000153    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5499b7b54d-qgj47"]
Jul 13 20:58:48.000433 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.000272    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.003352 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.003306    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 13 20:58:48.003463 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.003368    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 13 20:58:48.003664 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.003647    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 13 20:58:48.003805 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.003780    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g9hsd\""
Jul 13 20:58:48.003916 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.003845    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 13 20:58:48.003916 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.003855    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 13 20:58:48.004215 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.004194    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f2io6a5ob21ep\""
Jul 13 20:58:48.135186 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135147    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.135384 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135210    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-grpc-tls\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.135384 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135262    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.135384 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135290    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-tls\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.135384 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135307    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tgxj\" (UniqueName: \"kubernetes.io/projected/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-kube-api-access-4tgxj\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.135384 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135363    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.135659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135390    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-metrics-client-ca\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.135659 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.135420    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237035    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237113    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-tls\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237142    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tgxj\" (UniqueName: \"kubernetes.io/projected/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-kube-api-access-4tgxj\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237203    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237233    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-metrics-client-ca\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237265    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.238784 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.237909    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-grpc-tls\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.239198 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.238882    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-metrics-client-ca\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.242921 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.242878    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.243027 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.242991    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.245586 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.245547    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-grpc-tls\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.247046 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.247005    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.247417 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.247180    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-tls\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.247417 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.247320    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.247837 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.247799    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tgxj\" (UniqueName: \"kubernetes.io/projected/ae2bb4da-7bcb-4ea8-bed9-387890bc3f24-kube-api-access-4tgxj\") pod \"thanos-querier-5499b7b54d-qgj47\" (UID: \"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24\") " pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.312399 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.312286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:48.362055 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.362024    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 20:58:48.365704 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:48.365663    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf706187_8a93_4845_b443_d4f4838cf282.slice/crio-f87c2d4d8e41d72f4d4141dd1337afd11eb0d8dc1ad5de1fe5492a279c70bc81 WatchSource:0}: Error finding container f87c2d4d8e41d72f4d4141dd1337afd11eb0d8dc1ad5de1fe5492a279c70bc81: Status 404 returned error can't find the container with id f87c2d4d8e41d72f4d4141dd1337afd11eb0d8dc1ad5de1fe5492a279c70bc81
Jul 13 20:58:48.454942 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.454913    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5499b7b54d-qgj47"]
Jul 13 20:58:48.458428 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:48.458405    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2bb4da_7bcb_4ea8_bed9_387890bc3f24.slice/crio-92bc477c2ba747a8d95c1a2670c5979639172ac3c3e3277c90e905ad66313e69 WatchSource:0}: Error finding container 92bc477c2ba747a8d95c1a2670c5979639172ac3c3e3277c90e905ad66313e69: Status 404 returned error can't find the container with id 92bc477c2ba747a8d95c1a2670c5979639172ac3c3e3277c90e905ad66313e69
Jul 13 20:58:48.484700 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.484669    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4fxfq" event={"ID":"326c721f-7a82-4669-b58c-153c8ac68f50","Type":"ContainerStarted","Data":"0a666a01385f5f23dfa6aa1086648c5caf5c00aee61f63627a83f61b8804e9b6"}
Jul 13 20:58:48.484819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.484730    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4fxfq" event={"ID":"326c721f-7a82-4669-b58c-153c8ac68f50","Type":"ContainerStarted","Data":"fa1084f07f815d953f14128de75f27433adc667c965de23d5e090259b66e6d77"}
Jul 13 20:58:48.484819 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.484764    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:58:48.486135 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.486113    2556 generic.go:356] "Generic (PLEG): container finished" podID="acefebc3-f576-4baa-9913-d83d1e381678" containerID="5c6eabda46893fe3a0f0f7522d573070f04b9d8c8a0854284b69d9684d190522" exitCode=0
Jul 13 20:58:48.486238 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.486182    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgzt9" event={"ID":"acefebc3-f576-4baa-9913-d83d1e381678","Type":"ContainerDied","Data":"5c6eabda46893fe3a0f0f7522d573070f04b9d8c8a0854284b69d9684d190522"}
Jul 13 20:58:48.487292 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.487269    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" event={"ID":"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24","Type":"ContainerStarted","Data":"92bc477c2ba747a8d95c1a2670c5979639172ac3c3e3277c90e905ad66313e69"}
Jul 13 20:58:48.489487 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.489431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj" event={"ID":"9f1c8eac-1876-4340-8b83-feccc327e79a","Type":"ContainerStarted","Data":"abf625e68d02cda1a218a8fa7ff07aaaaf1a1cfea368cacc889cbf60d5dd2b25"}
Jul 13 20:58:48.490595 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.490577    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerStarted","Data":"f87c2d4d8e41d72f4d4141dd1337afd11eb0d8dc1ad5de1fe5492a279c70bc81"}
Jul 13 20:58:48.502123 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.502082    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4fxfq" podStartSLOduration=252.286413105 podStartE2EDuration="4m14.502069442s" totalImagesPullingTime="2.215656337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:48.500634554 +0000 UTC m=+289.438838902" watchObservedRunningTime="2026-07-13 20:58:48.502069442 +0000 UTC m=+289.440273752"
Jul 13 20:58:48.536630 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:48.536592    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-c97lj" podStartSLOduration=1.977702012 podStartE2EDuration="3.536579521s" totalImagesPullingTime="1.558877509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:48.535912681 +0000 UTC m=+289.474117010" watchObservedRunningTime="2026-07-13 20:58:48.536579521 +0000 UTC m=+289.474783833"
Jul 13 20:58:49.032462 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:49.032418    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:49.032919 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:49.032541    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:49.037961 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:49.037930    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:49.498218 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:49.498154    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgzt9" event={"ID":"acefebc3-f576-4baa-9913-d83d1e381678","Type":"ContainerStarted","Data":"49babac9ea606c9b9c341b943c3b8f56a13bab158677267557f4971bd6d3c6c8"}
Jul 13 20:58:49.498218 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:49.498219    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgzt9" event={"ID":"acefebc3-f576-4baa-9913-d83d1e381678","Type":"ContainerStarted","Data":"ea5f3758f6c801cc6909e91cca855fa6c19052402e5aa4a2497c0874e1c59855"}
Jul 13 20:58:49.504566 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:49.504540    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:58:49.520534 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:49.520484    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wgzt9" podStartSLOduration=2.544527709 podStartE2EDuration="4.520464732s" totalImagesPullingTime="1.899833329s" totalInitContainerRuntime="76.103694ms" isStatefulPod=true podCreationTimestamp="2026-07-13 20:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:49.518483565 +0000 UTC m=+290.456687879" watchObservedRunningTime="2026-07-13 20:58:49.520464732 +0000 UTC m=+290.458669044"
Jul 13 20:58:50.504697 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.504656    2556 generic.go:356] "Generic (PLEG): container finished" podID="af706187-8a93-4845-b443-d4f4838cf282" containerID="de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404" exitCode=0
Jul 13 20:58:50.504962 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.504733    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404"}
Jul 13 20:58:50.506665 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.506646    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" event={"ID":"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24","Type":"ContainerStarted","Data":"8ab305fa477e5d333f314eef8e59f567f8f25f79a9908bf7b0e2e16b3a189240"}
Jul 13 20:58:50.506755 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.506695    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" event={"ID":"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24","Type":"ContainerStarted","Data":"56052a4f06e2f422e28244624d255f300666a47ebd8f126dadc87038e1cfb784"}
Jul 13 20:58:50.715229 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.715196    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5c9677f5-tx7lc"]
Jul 13 20:58:50.717505 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.717487    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.724867 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.724848    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 13 20:58:50.728870 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.728850    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5c9677f5-tx7lc"]
Jul 13 20:58:50.867212 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.867176    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-serving-cert\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.867212 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.867219    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-oauth-serving-cert\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.867443 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.867238    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r74ls\" (UniqueName: \"kubernetes.io/projected/14221df1-7cd4-4636-a2b9-73488739fef1-kube-api-access-r74ls\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.867443 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.867325    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-trusted-ca-bundle\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.867443 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.867420    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-service-ca\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.867577 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.867462    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-console-config\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.867577 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.867528    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-oauth-config\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969008 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.968923    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-serving-cert\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969008 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.968966    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-oauth-serving-cert\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969008 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.968985    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r74ls\" (UniqueName: \"kubernetes.io/projected/14221df1-7cd4-4636-a2b9-73488739fef1-kube-api-access-r74ls\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969271 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969018    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-trusted-ca-bundle\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969271 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969055    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-service-ca\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969271 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969088    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-console-config\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969271 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969255    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-oauth-config\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969824 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969797    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-oauth-serving-cert\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969950 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969855    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-service-ca\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.969950 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969873    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-console-config\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.970072 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.969998    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-trusted-ca-bundle\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.971481 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.971455    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-serving-cert\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.971669 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.971647    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-oauth-config\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:50.976388 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:50.976372    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r74ls\" (UniqueName: \"kubernetes.io/projected/14221df1-7cd4-4636-a2b9-73488739fef1-kube-api-access-r74ls\") pod \"console-5c5c9677f5-tx7lc\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") " pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:51.026800 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:51.026768    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:58:51.159927 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:51.159902    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5c9677f5-tx7lc"]
Jul 13 20:58:51.162112 ip-10-0-138-146 kubenswrapper[2556]: W0713 20:58:51.162046    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14221df1_7cd4_4636_a2b9_73488739fef1.slice/crio-8c57bc71de33c5ee00252c62307ce3d78b66803819e8310dbdc7a9ff139e6e15 WatchSource:0}: Error finding container 8c57bc71de33c5ee00252c62307ce3d78b66803819e8310dbdc7a9ff139e6e15: Status 404 returned error can't find the container with id 8c57bc71de33c5ee00252c62307ce3d78b66803819e8310dbdc7a9ff139e6e15
Jul 13 20:58:51.513900 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:51.513872    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9677f5-tx7lc" event={"ID":"14221df1-7cd4-4636-a2b9-73488739fef1","Type":"ContainerStarted","Data":"e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5"}
Jul 13 20:58:51.514238 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:51.513930    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9677f5-tx7lc" event={"ID":"14221df1-7cd4-4636-a2b9-73488739fef1","Type":"ContainerStarted","Data":"8c57bc71de33c5ee00252c62307ce3d78b66803819e8310dbdc7a9ff139e6e15"}
Jul 13 20:58:51.517607 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:51.517582    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" event={"ID":"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24","Type":"ContainerStarted","Data":"ebe0352a1f34c1e11a15c118e3ac01927a998c871374f2c74b2658136dcde6d5"}
Jul 13 20:58:51.517695 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:51.517653    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" event={"ID":"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24","Type":"ContainerStarted","Data":"90dfd008d370974e4fa645d4bd3388d51b01c9f25c51e33d2daa44a623b74388"}
Jul 13 20:58:51.531715 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:51.531625    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c5c9677f5-tx7lc" podStartSLOduration=1.531605248 podStartE2EDuration="1.531605248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:51.530509319 +0000 UTC m=+292.468713631" watchObservedRunningTime="2026-07-13 20:58:51.531605248 +0000 UTC m=+292.469809561"
Jul 13 20:58:52.526566 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:52.526536    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" event={"ID":"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24","Type":"ContainerStarted","Data":"b92702f4b8ce455ee56f4bad50dff75f6cd6cb702e99b724cff1def7a5480e52"}
Jul 13 20:58:52.526976 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:52.526617    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" event={"ID":"ae2bb4da-7bcb-4ea8-bed9-387890bc3f24","Type":"ContainerStarted","Data":"69d2c651665d67760beabf14a5c08af8732da4fa3bdf3165747dee76c81085fc"}
Jul 13 20:58:52.526976 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:52.526765    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:52.528908 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:52.528877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerStarted","Data":"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"}
Jul 13 20:58:52.529011 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:52.528951    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerStarted","Data":"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"}
Jul 13 20:58:52.551620 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:52.551571    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47" podStartSLOduration=2.796083354 podStartE2EDuration="5.551552736s" totalImagesPullingTime="2.755469382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:52.54957213 +0000 UTC m=+293.487776441" watchObservedRunningTime="2026-07-13 20:58:52.551552736 +0000 UTC m=+293.489757048"
Jul 13 20:58:53.536766 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:53.536733    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerStarted","Data":"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"}
Jul 13 20:58:53.537117 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:53.536821    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerStarted","Data":"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"}
Jul 13 20:58:53.537117 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:53.536853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerStarted","Data":"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"}
Jul 13 20:58:53.537117 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:53.536879    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerStarted","Data":"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"}
Jul 13 20:58:53.566557 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:53.566509    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.467112645 podStartE2EDuration="7.566495224s" totalImagesPullingTime="3.081193407s" totalInitContainerRuntime="18.189172ms" isStatefulPod=false podCreationTimestamp="2026-07-13 20:58:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:58:53.564654159 +0000 UTC m=+294.502858483" watchObservedRunningTime="2026-07-13 20:58:53.566495224 +0000 UTC m=+294.504699534"
Jul 13 20:58:56.442607 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:56.442580    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:58:56.598207 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:58:56.598175    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-grcrp" podUID="de4035ca-3984-435e-8387-217677bcc954"
Jul 13 20:58:58.501432 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:58.501403    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4fxfq"
Jul 13 20:58:58.543332 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:58.543304    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5499b7b54d-qgj47"
Jul 13 20:58:59.458070 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:59.458040    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 20:58:59.458230 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:59.458150    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 20:58:59.469975 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:58:59.469955    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 20:59:01.027461 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.027426    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:59:01.028861 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.027484    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:59:01.032057 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.032036    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:59:01.457596 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.457536    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" podUID="41e2a709-c033-4b04-99ef-c5876668a5b1" containerName="registry" containerID="cri-o://ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f" gracePeriod=30
Jul 13 20:59:01.572254 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.572228    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 20:59:01.616544 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.616514    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c54bf59b6-r2gnr"]
Jul 13 20:59:01.725641 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.725619    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:59:01.764058 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764034    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-image-registry-private-configuration\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764228 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764074    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764228 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764107    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-certificates\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764228 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764149    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-bound-sa-token\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764228 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764191    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-installation-pull-secrets\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764458 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764248    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-trusted-ca\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764458 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764304    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41e2a709-c033-4b04-99ef-c5876668a5b1-ca-trust-extracted\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764458 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764357    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn5xb\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-kube-api-access-gn5xb\") pod \"41e2a709-c033-4b04-99ef-c5876668a5b1\" (UID: \"41e2a709-c033-4b04-99ef-c5876668a5b1\") "
Jul 13 20:59:01.764649 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.764616    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:59:01.765099 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.765027    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:59:01.767029 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.766939    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:59:01.767029 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.766987    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:59:01.767191 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.767045    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:59:01.767191 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.767062    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:59:01.767191 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.767169    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-kube-api-access-gn5xb" (OuterVolumeSpecName: "kube-api-access-gn5xb") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "kube-api-access-gn5xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:59:01.773502 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.773476    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41e2a709-c033-4b04-99ef-c5876668a5b1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "41e2a709-c033-4b04-99ef-c5876668a5b1" (UID: "41e2a709-c033-4b04-99ef-c5876668a5b1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 20:59:01.864972 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.864945    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-installation-pull-secrets\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:01.864972 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.864969    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-trusted-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:01.864972 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.864978    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41e2a709-c033-4b04-99ef-c5876668a5b1-ca-trust-extracted\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:01.865174 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.864987    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn5xb\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-kube-api-access-gn5xb\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:01.865174 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.864998    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41e2a709-c033-4b04-99ef-c5876668a5b1-image-registry-private-configuration\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:01.865174 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.865006    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-tls\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:01.865174 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.865014    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41e2a709-c033-4b04-99ef-c5876668a5b1-registry-certificates\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:01.865174 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:01.865022    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41e2a709-c033-4b04-99ef-c5876668a5b1-bound-sa-token\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:02.573727 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.573690    2556 generic.go:356] "Generic (PLEG): container finished" podID="41e2a709-c033-4b04-99ef-c5876668a5b1" containerID="ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f" exitCode=0
Jul 13 20:59:02.574117 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.573733    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk"
Jul 13 20:59:02.574117 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.573773    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" event={"ID":"41e2a709-c033-4b04-99ef-c5876668a5b1","Type":"ContainerDied","Data":"ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f"}
Jul 13 20:59:02.574117 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.573821    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd4c58db5-r9djk" event={"ID":"41e2a709-c033-4b04-99ef-c5876668a5b1","Type":"ContainerDied","Data":"e75d0424ab69635a8e645784744e262973038ea49873e54a54feec9169df5ea8"}
Jul 13 20:59:02.574117 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.573842    2556 scope.go:117] "RemoveContainer" containerID="ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f"
Jul 13 20:59:02.582788 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.582771    2556 scope.go:117] "RemoveContainer" containerID="ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f"
Jul 13 20:59:02.583051 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:59:02.583033    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f\": container with ID starting with ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f not found: ID does not exist" containerID="ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f"
Jul 13 20:59:02.583136 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.583060    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f"} err="failed to get container status \"ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f\": rpc error: code = NotFound desc = could not find container \"ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f\": container with ID starting with ef2114594bec939d72bbd2c5da64837bc0a0478c225a1f001ce0cb189e34564f not found: ID does not exist"
Jul 13 20:59:02.594287 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.594263    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bd4c58db5-r9djk"]
Jul 13 20:59:02.597822 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:02.597803    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6bd4c58db5-r9djk"]
Jul 13 20:59:03.602103 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:03.602068    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41e2a709-c033-4b04-99ef-c5876668a5b1" path="/var/lib/kubelet/pods/41e2a709-c033-4b04-99ef-c5876668a5b1/volumes"
Jul 13 20:59:11.602207 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:11.602179    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 20:59:23.656881 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:23.656849    2556 generic.go:356] "Generic (PLEG): container finished" podID="7518b6cb-fdc2-46b6-934c-e73079d7a0d2" containerID="905c94624f6980a26a0e0b805acd92caedb80bbf86a5d2d55a7df28a1d03e66c" exitCode=0
Jul 13 20:59:23.657273 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:23.656923    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z" event={"ID":"7518b6cb-fdc2-46b6-934c-e73079d7a0d2","Type":"ContainerDied","Data":"905c94624f6980a26a0e0b805acd92caedb80bbf86a5d2d55a7df28a1d03e66c"}
Jul 13 20:59:23.657273 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:23.657202    2556 scope.go:117] "RemoveContainer" containerID="905c94624f6980a26a0e0b805acd92caedb80bbf86a5d2d55a7df28a1d03e66c"
Jul 13 20:59:24.661975 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:24.661942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-mvx2z" event={"ID":"7518b6cb-fdc2-46b6-934c-e73079d7a0d2","Type":"ContainerStarted","Data":"b515efe199e06d54241ac4e426b23cce5542279af8eb317a164dec3b15d7a9fb"}
Jul 13 20:59:26.639846 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.639796    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c54bf59b6-r2gnr" podUID="6dfedac0-64d5-4664-ac18-755e5239120a" containerName="console" containerID="cri-o://772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd" gracePeriod=15
Jul 13 20:59:26.884195 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.884168    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:59:26.981157 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981051    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-serving-cert\") pod \"6dfedac0-64d5-4664-ac18-755e5239120a\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") "
Jul 13 20:59:26.981157 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981104    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76lp8\" (UniqueName: \"kubernetes.io/projected/6dfedac0-64d5-4664-ac18-755e5239120a-kube-api-access-76lp8\") pod \"6dfedac0-64d5-4664-ac18-755e5239120a\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") "
Jul 13 20:59:26.981157 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981121    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-service-ca\") pod \"6dfedac0-64d5-4664-ac18-755e5239120a\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") "
Jul 13 20:59:26.981157 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981156    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-oauth-serving-cert\") pod \"6dfedac0-64d5-4664-ac18-755e5239120a\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") "
Jul 13 20:59:26.981763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981211    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-oauth-config\") pod \"6dfedac0-64d5-4664-ac18-755e5239120a\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") "
Jul 13 20:59:26.981763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981279    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-console-config\") pod \"6dfedac0-64d5-4664-ac18-755e5239120a\" (UID: \"6dfedac0-64d5-4664-ac18-755e5239120a\") "
Jul 13 20:59:26.981763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981571    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-service-ca" (OuterVolumeSpecName: "service-ca") pod "6dfedac0-64d5-4664-ac18-755e5239120a" (UID: "6dfedac0-64d5-4664-ac18-755e5239120a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:59:26.981763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981586    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6dfedac0-64d5-4664-ac18-755e5239120a" (UID: "6dfedac0-64d5-4664-ac18-755e5239120a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:59:26.981763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981664    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-console-config" (OuterVolumeSpecName: "console-config") pod "6dfedac0-64d5-4664-ac18-755e5239120a" (UID: "6dfedac0-64d5-4664-ac18-755e5239120a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 20:59:26.981763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981680    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-service-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:26.981763 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.981697    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-oauth-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:26.983368 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.983330    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dfedac0-64d5-4664-ac18-755e5239120a-kube-api-access-76lp8" (OuterVolumeSpecName: "kube-api-access-76lp8") pod "6dfedac0-64d5-4664-ac18-755e5239120a" (UID: "6dfedac0-64d5-4664-ac18-755e5239120a"). InnerVolumeSpecName "kube-api-access-76lp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 20:59:26.983717 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.983696    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6dfedac0-64d5-4664-ac18-755e5239120a" (UID: "6dfedac0-64d5-4664-ac18-755e5239120a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:59:26.983775 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:26.983726    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6dfedac0-64d5-4664-ac18-755e5239120a" (UID: "6dfedac0-64d5-4664-ac18-755e5239120a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 20:59:27.082214 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.082179    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:27.082214 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.082207    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76lp8\" (UniqueName: \"kubernetes.io/projected/6dfedac0-64d5-4664-ac18-755e5239120a-kube-api-access-76lp8\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:27.082214 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.082216    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dfedac0-64d5-4664-ac18-755e5239120a-console-oauth-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:27.082449 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.082224    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dfedac0-64d5-4664-ac18-755e5239120a-console-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 20:59:27.674492 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.674458    2556 generic.go:356] "Generic (PLEG): container finished" podID="6dfedac0-64d5-4664-ac18-755e5239120a" containerID="772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd" exitCode=0
Jul 13 20:59:27.674860 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.674504    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c54bf59b6-r2gnr"
Jul 13 20:59:27.674860 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.674503    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c54bf59b6-r2gnr" event={"ID":"6dfedac0-64d5-4664-ac18-755e5239120a","Type":"ContainerDied","Data":"772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd"}
Jul 13 20:59:27.674860 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.674631    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c54bf59b6-r2gnr" event={"ID":"6dfedac0-64d5-4664-ac18-755e5239120a","Type":"ContainerDied","Data":"c5dde5c4ff383c205e005d4d5fc0fc22bc60650a76c0fafede8a16589647a06b"}
Jul 13 20:59:27.674860 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.674661    2556 scope.go:117] "RemoveContainer" containerID="772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd"
Jul 13 20:59:27.683223 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.683197    2556 scope.go:117] "RemoveContainer" containerID="772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd"
Jul 13 20:59:27.683541 ip-10-0-138-146 kubenswrapper[2556]: E0713 20:59:27.683522    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd\": container with ID starting with 772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd not found: ID does not exist" containerID="772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd"
Jul 13 20:59:27.683618 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.683549    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd"} err="failed to get container status \"772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd\": rpc error: code = NotFound desc = could not find container \"772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd\": container with ID starting with 772ddbeb8b33bf5e06368ee48fba905c185cdd2b2a7f8bda6ebde5470af111fd not found: ID does not exist"
Jul 13 20:59:27.689964 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.689945    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c54bf59b6-r2gnr"]
Jul 13 20:59:27.693924 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:27.693899    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c54bf59b6-r2gnr"]
Jul 13 20:59:28.681501 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:28.681475    2556 generic.go:356] "Generic (PLEG): container finished" podID="f7e89441-6447-4b44-b1e3-b3a8ead9aa13" containerID="b64f34ae09b5bb9ad1ece26b8594fb76a3637ccb049372d9c89f7583c3ecd67b" exitCode=0
Jul 13 20:59:28.681859 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:28.681528    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9" event={"ID":"f7e89441-6447-4b44-b1e3-b3a8ead9aa13","Type":"ContainerDied","Data":"b64f34ae09b5bb9ad1ece26b8594fb76a3637ccb049372d9c89f7583c3ecd67b"}
Jul 13 20:59:28.681859 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:28.681817    2556 scope.go:117] "RemoveContainer" containerID="b64f34ae09b5bb9ad1ece26b8594fb76a3637ccb049372d9c89f7583c3ecd67b"
Jul 13 20:59:29.602722 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:29.602690    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dfedac0-64d5-4664-ac18-755e5239120a" path="/var/lib/kubelet/pods/6dfedac0-64d5-4664-ac18-755e5239120a/volumes"
Jul 13 20:59:29.687103 ip-10-0-138-146 kubenswrapper[2556]: I0713 20:59:29.687073    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bkql9" event={"ID":"f7e89441-6447-4b44-b1e3-b3a8ead9aa13","Type":"ContainerStarted","Data":"3b0c30bcaf05f3858aebd53c3c26ea4a381331e020fc21c8622daabf1c7b9b7a"}
Jul 13 21:00:06.194139 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.194105    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:00:06.194618 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.194473    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="alertmanager" containerID="cri-o://bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e" gracePeriod=120
Jul 13 21:00:06.194618 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.194542    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-web" containerID="cri-o://c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa" gracePeriod=120
Jul 13 21:00:06.194618 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.194541    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-metric" containerID="cri-o://c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86" gracePeriod=120
Jul 13 21:00:06.194618 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.194573    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="prom-label-proxy" containerID="cri-o://63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8" gracePeriod=120
Jul 13 21:00:06.194618 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.194616    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy" containerID="cri-o://220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec" gracePeriod=120
Jul 13 21:00:06.194874 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.194572    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="config-reloader" containerID="cri-o://554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965" gracePeriod=120
Jul 13 21:00:06.851266 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851236    2556 generic.go:356] "Generic (PLEG): container finished" podID="af706187-8a93-4845-b443-d4f4838cf282" containerID="63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8" exitCode=0
Jul 13 21:00:06.851266 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851257    2556 generic.go:356] "Generic (PLEG): container finished" podID="af706187-8a93-4845-b443-d4f4838cf282" containerID="220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec" exitCode=0
Jul 13 21:00:06.851266 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851263    2556 generic.go:356] "Generic (PLEG): container finished" podID="af706187-8a93-4845-b443-d4f4838cf282" containerID="554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965" exitCode=0
Jul 13 21:00:06.851266 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851269    2556 generic.go:356] "Generic (PLEG): container finished" podID="af706187-8a93-4845-b443-d4f4838cf282" containerID="bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e" exitCode=0
Jul 13 21:00:06.851549 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851307    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"}
Jul 13 21:00:06.851549 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851382    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"}
Jul 13 21:00:06.851549 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851412    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"}
Jul 13 21:00:06.851549 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:06.851436    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"}
Jul 13 21:00:07.436291 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.436270    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:07.518373 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518290    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-web-config\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518373 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518331    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-main-db\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518378    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-web\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518395    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-config-volume\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518412    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518432    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518470    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29vhk\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-kube-api-access-29vhk\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518502    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-metrics-client-ca\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518526    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-cluster-tls-config\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518560    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-config-out\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518590    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518611    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-metric\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518644    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-tls-assets\") pod \"af706187-8a93-4845-b443-d4f4838cf282\" (UID: \"af706187-8a93-4845-b443-d4f4838cf282\") "
Jul 13 21:00:07.518959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.518773    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:00:07.520739 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.520707    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:00:07.521558 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.521401    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:07.521558 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.521484    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:07.521558 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.521524    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-config-volume" (OuterVolumeSpecName: "config-volume") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:07.522122 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.522085    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-main-db\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.522811 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.522634    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:00:07.525676 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.525652    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-config-out" (OuterVolumeSpecName: "config-out") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:00:07.528132 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.527988    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:07.528265 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.528156    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:00:07.528533 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.528398    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-kube-api-access-29vhk" (OuterVolumeSpecName: "kube-api-access-29vhk") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "kube-api-access-29vhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:00:07.528919 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.528883    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:07.532349 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.532312    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:07.536229 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.536209    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-web-config" (OuterVolumeSpecName: "web-config") pod "af706187-8a93-4845-b443-d4f4838cf282" (UID: "af706187-8a93-4845-b443-d4f4838cf282"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:07.623354 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623309    2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-web-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623354 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623335    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623365    2556 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-config-volume\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623378    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623390    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623403    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29vhk\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-kube-api-access-29vhk\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623415    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af706187-8a93-4845-b443-d4f4838cf282-metrics-client-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623427    2556 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-cluster-tls-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623439    2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af706187-8a93-4845-b443-d4f4838cf282-config-out\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623451    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-main-tls\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623463    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af706187-8a93-4845-b443-d4f4838cf282-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.623535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.623476    2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af706187-8a93-4845-b443-d4f4838cf282-tls-assets\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:07.857881 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.857801    2556 generic.go:356] "Generic (PLEG): container finished" podID="af706187-8a93-4845-b443-d4f4838cf282" containerID="c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86" exitCode=0
Jul 13 21:00:07.857881 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.857824    2556 generic.go:356] "Generic (PLEG): container finished" podID="af706187-8a93-4845-b443-d4f4838cf282" containerID="c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa" exitCode=0
Jul 13 21:00:07.857881 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.857867    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"}
Jul 13 21:00:07.858087 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.857886    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:07.858087 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.857942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"}
Jul 13 21:00:07.858087 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.857971    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af706187-8a93-4845-b443-d4f4838cf282","Type":"ContainerDied","Data":"f87c2d4d8e41d72f4d4141dd1337afd11eb0d8dc1ad5de1fe5492a279c70bc81"}
Jul 13 21:00:07.858087 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.858016    2556 scope.go:117] "RemoveContainer" containerID="63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"
Jul 13 21:00:07.865472 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.865458    2556 scope.go:117] "RemoveContainer" containerID="c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"
Jul 13 21:00:07.872426 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.872409    2556 scope.go:117] "RemoveContainer" containerID="220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"
Jul 13 21:00:07.878527 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.878509    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:00:07.879321 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.879304    2556 scope.go:117] "RemoveContainer" containerID="c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"
Jul 13 21:00:07.883391 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.883373    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:00:07.885920 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.885907    2556 scope.go:117] "RemoveContainer" containerID="554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"
Jul 13 21:00:07.892363 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.892332    2556 scope.go:117] "RemoveContainer" containerID="bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"
Jul 13 21:00:07.898925 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.898908    2556 scope.go:117] "RemoveContainer" containerID="de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404"
Jul 13 21:00:07.905427 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.905409    2556 scope.go:117] "RemoveContainer" containerID="63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"
Jul 13 21:00:07.905669 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:07.905652    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8\": container with ID starting with 63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8 not found: ID does not exist" containerID="63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"
Jul 13 21:00:07.905722 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.905676    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"} err="failed to get container status \"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8\": rpc error: code = NotFound desc = could not find container \"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8\": container with ID starting with 63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8 not found: ID does not exist"
Jul 13 21:00:07.905722 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.905693    2556 scope.go:117] "RemoveContainer" containerID="c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"
Jul 13 21:00:07.905898 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:07.905883    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86\": container with ID starting with c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86 not found: ID does not exist" containerID="c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"
Jul 13 21:00:07.905942 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.905903    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"} err="failed to get container status \"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86\": rpc error: code = NotFound desc = could not find container \"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86\": container with ID starting with c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86 not found: ID does not exist"
Jul 13 21:00:07.905942 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.905917    2556 scope.go:117] "RemoveContainer" containerID="220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"
Jul 13 21:00:07.906112 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:07.906095    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec\": container with ID starting with 220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec not found: ID does not exist" containerID="220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"
Jul 13 21:00:07.906176 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906121    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"} err="failed to get container status \"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec\": rpc error: code = NotFound desc = could not find container \"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec\": container with ID starting with 220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec not found: ID does not exist"
Jul 13 21:00:07.906176 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906144    2556 scope.go:117] "RemoveContainer" containerID="c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"
Jul 13 21:00:07.906425 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:07.906394    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa\": container with ID starting with c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa not found: ID does not exist" containerID="c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"
Jul 13 21:00:07.906425 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906413    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"} err="failed to get container status \"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa\": rpc error: code = NotFound desc = could not find container \"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa\": container with ID starting with c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa not found: ID does not exist"
Jul 13 21:00:07.906425 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906425    2556 scope.go:117] "RemoveContainer" containerID="554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"
Jul 13 21:00:07.906614 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:07.906599    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965\": container with ID starting with 554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965 not found: ID does not exist" containerID="554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"
Jul 13 21:00:07.906653 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906618    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"} err="failed to get container status \"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965\": rpc error: code = NotFound desc = could not find container \"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965\": container with ID starting with 554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965 not found: ID does not exist"
Jul 13 21:00:07.906653 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906640    2556 scope.go:117] "RemoveContainer" containerID="bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"
Jul 13 21:00:07.906814 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:07.906799    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e\": container with ID starting with bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e not found: ID does not exist" containerID="bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"
Jul 13 21:00:07.906849 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906816    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"} err="failed to get container status \"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e\": rpc error: code = NotFound desc = could not find container \"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e\": container with ID starting with bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e not found: ID does not exist"
Jul 13 21:00:07.906849 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906828    2556 scope.go:117] "RemoveContainer" containerID="de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404"
Jul 13 21:00:07.906989 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:07.906974    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404\": container with ID starting with de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404 not found: ID does not exist" containerID="de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404"
Jul 13 21:00:07.907029 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.906990    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404"} err="failed to get container status \"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404\": rpc error: code = NotFound desc = could not find container \"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404\": container with ID starting with de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404 not found: ID does not exist"
Jul 13 21:00:07.907029 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907001    2556 scope.go:117] "RemoveContainer" containerID="63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"
Jul 13 21:00:07.907173 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907159    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8"} err="failed to get container status \"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8\": rpc error: code = NotFound desc = could not find container \"63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8\": container with ID starting with 63177f33c4c8edcec28b73886194bafc9ccd906e1afd39aede63562978fdb5c8 not found: ID does not exist"
Jul 13 21:00:07.907215 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907173    2556 scope.go:117] "RemoveContainer" containerID="c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"
Jul 13 21:00:07.907389 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907366    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86"} err="failed to get container status \"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86\": rpc error: code = NotFound desc = could not find container \"c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86\": container with ID starting with c14d727f887cd911421d3e993f7b212f3ec5a172f539212c4ff96e1688d33f86 not found: ID does not exist"
Jul 13 21:00:07.907438 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907392    2556 scope.go:117] "RemoveContainer" containerID="220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"
Jul 13 21:00:07.907628 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907614    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec"} err="failed to get container status \"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec\": rpc error: code = NotFound desc = could not find container \"220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec\": container with ID starting with 220c060bec8c3396c8082b8fd9bd6f3c3ad33225a68fb94c924f82a2abd1bdec not found: ID does not exist"
Jul 13 21:00:07.907669 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907628    2556 scope.go:117] "RemoveContainer" containerID="c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"
Jul 13 21:00:07.907826 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907810    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa"} err="failed to get container status \"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa\": rpc error: code = NotFound desc = could not find container \"c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa\": container with ID starting with c26042518cb121d9e8b7a426b92b5f78ebc938261430094f7d809361d21439aa not found: ID does not exist"
Jul 13 21:00:07.907870 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.907826    2556 scope.go:117] "RemoveContainer" containerID="554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"
Jul 13 21:00:07.908069 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.908050    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965"} err="failed to get container status \"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965\": rpc error: code = NotFound desc = could not find container \"554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965\": container with ID starting with 554392af20004ba0302d20dba6b51ccca84ade5770a00ded78a3f6f596a3b965 not found: ID does not exist"
Jul 13 21:00:07.908106 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.908069    2556 scope.go:117] "RemoveContainer" containerID="bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"
Jul 13 21:00:07.908316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.908297    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e"} err="failed to get container status \"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e\": rpc error: code = NotFound desc = could not find container \"bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e\": container with ID starting with bc2a22607cab2c2bc892c0fff65d8d53bf4f4c7d60ee1896eb829d582c64455e not found: ID does not exist"
Jul 13 21:00:07.908413 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.908317    2556 scope.go:117] "RemoveContainer" containerID="de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404"
Jul 13 21:00:07.908683 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.908662    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404"} err="failed to get container status \"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404\": rpc error: code = NotFound desc = could not find container \"de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404\": container with ID starting with de0b41ff3b37298626c3a820ea4f8e440bc277990ca3946c589bcca1bccb8404 not found: ID does not exist"
Jul 13 21:00:07.910190 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910173    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:00:07.910493 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910481    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="alertmanager"
Jul 13 21:00:07.910540 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910495    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="alertmanager"
Jul 13 21:00:07.910540 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910510    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dfedac0-64d5-4664-ac18-755e5239120a" containerName="console"
Jul 13 21:00:07.910540 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910517    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dfedac0-64d5-4664-ac18-755e5239120a" containerName="console"
Jul 13 21:00:07.910540 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910527    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-metric"
Jul 13 21:00:07.910540 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910533    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-metric"
Jul 13 21:00:07.910540 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910540    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="init-config-reloader"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910545    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="init-config-reloader"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910552    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41e2a709-c033-4b04-99ef-c5876668a5b1" containerName="registry"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910557    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="41e2a709-c033-4b04-99ef-c5876668a5b1" containerName="registry"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910564    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-web"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910568    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-web"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910579    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="config-reloader"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910588    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="config-reloader"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910596    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="prom-label-proxy"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910601    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="prom-label-proxy"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910607    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910612    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910665    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="41e2a709-c033-4b04-99ef-c5876668a5b1" containerName="registry"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910673    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-metric"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910680    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dfedac0-64d5-4664-ac18-755e5239120a" containerName="console"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910687    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="config-reloader"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910691    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy-web"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910696    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="kube-rbac-proxy"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910702    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="prom-label-proxy"
Jul 13 21:00:07.910706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.910708    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af706187-8a93-4845-b443-d4f4838cf282" containerName="alertmanager"
Jul 13 21:00:07.915572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.915557    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:07.918230 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918211    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 21:00:07.918373 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918309    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 21:00:07.918460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918376    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 21:00:07.918460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918387    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 21:00:07.918460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918447    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5jh9g\""
Jul 13 21:00:07.918630 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918375    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 21:00:07.918630 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918473    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 21:00:07.918630 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918501    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 21:00:07.918630 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.918606    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 21:00:07.925505 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.925437    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 21:00:07.927381 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:07.927325    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:00:08.025591 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025591 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025592    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkjsh\" (UniqueName: \"kubernetes.io/projected/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-kube-api-access-lkjsh\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025762 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025612    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025762 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025638    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025762 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025762 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025690    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025762 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025738    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025765    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025783    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-config-volume\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025798    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-config-out\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025827    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-web-config\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.025910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.025863    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126375 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126324    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126390    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126409    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126430    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126572 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126838 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126585    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-config-volume\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126838 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-config-out\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126838 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126641    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126838 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-web-config\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126838 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126691    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126838 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126739    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.126838 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.126767    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkjsh\" (UniqueName: \"kubernetes.io/projected/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-kube-api-access-lkjsh\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.127187 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.127161    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.127532 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.127509    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.129679 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129525    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.129679 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129561    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.129679 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129644    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.129889 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129745    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.129889 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129793    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-config-volume\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.129889 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129836    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-web-config\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.130052 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129959    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-config-out\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.130052 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.129991    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.130358 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.130316    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.131607 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.131589    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.134375 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.134358    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkjsh\" (UniqueName: \"kubernetes.io/projected/53590627-1cd2-43e9-a8bb-fc13ae7acf7a-kube-api-access-lkjsh\") pod \"alertmanager-main-0\" (UID: \"53590627-1cd2-43e9-a8bb-fc13ae7acf7a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.226367 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.226324    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:00:08.358133 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.358109    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:00:08.360360 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:00:08.360324    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53590627_1cd2_43e9_a8bb_fc13ae7acf7a.slice/crio-dbb171d0c13a979d2ce39d852c792c70c799371acb360c45d9b7d132579425d5 WatchSource:0}: Error finding container dbb171d0c13a979d2ce39d852c792c70c799371acb360c45d9b7d132579425d5: Status 404 returned error can't find the container with id dbb171d0c13a979d2ce39d852c792c70c799371acb360c45d9b7d132579425d5
Jul 13 21:00:08.864201 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.864164    2556 generic.go:356] "Generic (PLEG): container finished" podID="53590627-1cd2-43e9-a8bb-fc13ae7acf7a" containerID="5e01212b00ccf4bcd861d0baacd4a7614c98b09a4b87cd1378b813738ecee274" exitCode=0
Jul 13 21:00:08.864556 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.864205    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerDied","Data":"5e01212b00ccf4bcd861d0baacd4a7614c98b09a4b87cd1378b813738ecee274"}
Jul 13 21:00:08.864556 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:08.864264    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerStarted","Data":"dbb171d0c13a979d2ce39d852c792c70c799371acb360c45d9b7d132579425d5"}
Jul 13 21:00:09.602601 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.602526    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af706187-8a93-4845-b443-d4f4838cf282" path="/var/lib/kubelet/pods/af706187-8a93-4845-b443-d4f4838cf282/volumes"
Jul 13 21:00:09.871750 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.871717    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerStarted","Data":"427019241ccab4454d7d123e6849159b4abbd4963fa6e12decf5343868a836bc"}
Jul 13 21:00:09.872193 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.871784    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerStarted","Data":"c1c69d1296a7bf4756a48498e0b001d2ef4c0c274234ddbdad7e38230f80cfb2"}
Jul 13 21:00:09.872193 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.871810    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerStarted","Data":"727acb003599173efedecadf2261c66a7bcb8ae934e9a998403d4b18017b6e37"}
Jul 13 21:00:09.872193 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.871833    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerStarted","Data":"acd669af1dce9dd223db5591b084196c06826b35c4ebe9e80f422b1e6e951c93"}
Jul 13 21:00:09.872193 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.871857    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerStarted","Data":"5aa757bdb49c6bf9141d412e1752cc57c8c7fbe09efd9ab0d39314e76d925a74"}
Jul 13 21:00:09.872193 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.871880    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"53590627-1cd2-43e9-a8bb-fc13ae7acf7a","Type":"ContainerStarted","Data":"f43b27a524dcef061e9e3b2f2aacd6b2d04bf4ec2d792c2959061edcd3df7083"}
Jul 13 21:00:09.905190 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:09.905146    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.886768847 podStartE2EDuration="2.905132315s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.363468ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:09.904818444 +0000 UTC m=+370.843022757" watchObservedRunningTime="2026-07-13 21:00:09.905132315 +0000 UTC m=+370.843336626"
Jul 13 21:00:10.246793 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.246714    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5696968dc5-nxh7z"]
Jul 13 21:00:10.251587 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.251563    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.256287 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.256261    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 13 21:00:10.256412 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.256295    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 13 21:00:10.256412 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.256379    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 13 21:00:10.256531 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.256429    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 13 21:00:10.256637 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.256618    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 13 21:00:10.256769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.256676    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zx8ms\""
Jul 13 21:00:10.274737 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.274688    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 13 21:00:10.341466 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.341434    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5696968dc5-nxh7z"]
Jul 13 21:00:10.346755 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.346725    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.346872 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.346780    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-serving-certs-ca-bundle\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.346872 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.346857    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l67h\" (UniqueName: \"kubernetes.io/projected/c0156185-4b50-4b11-9332-fd09471fa634-kube-api-access-9l67h\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.346963 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.346893    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-secret-telemeter-client\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.346963 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.346932    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.347073 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.346972    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-metrics-client-ca\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.347073 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.347007    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-telemeter-client-tls\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.347142 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.347083    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-federate-client-tls\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447641 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447596    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-federate-client-tls\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447805 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447697    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447805 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447738    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-serving-certs-ca-bundle\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447805 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447787    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l67h\" (UniqueName: \"kubernetes.io/projected/c0156185-4b50-4b11-9332-fd09471fa634-kube-api-access-9l67h\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447983 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447820    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-secret-telemeter-client\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447983 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447983 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447883    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-metrics-client-ca\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.447983 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.447922    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-telemeter-client-tls\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.448597 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.448569    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-serving-certs-ca-bundle\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.448718 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.448662    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.448782 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.448757    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0156185-4b50-4b11-9332-fd09471fa634-metrics-client-ca\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.450319 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.450291    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-federate-client-tls\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.450453 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.450432    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-telemeter-client-tls\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.450708 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.450691    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.450790 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.450771    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c0156185-4b50-4b11-9332-fd09471fa634-secret-telemeter-client\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.455979 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.455950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l67h\" (UniqueName: \"kubernetes.io/projected/c0156185-4b50-4b11-9332-fd09471fa634-kube-api-access-9l67h\") pod \"telemeter-client-5696968dc5-nxh7z\" (UID: \"c0156185-4b50-4b11-9332-fd09471fa634\") " pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.561954 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.561869    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z"
Jul 13 21:00:10.711199 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.710888    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5696968dc5-nxh7z"]
Jul 13 21:00:10.713771 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:00:10.713746    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0156185_4b50_4b11_9332_fd09471fa634.slice/crio-b480b38026dedab69386e1826af2b1642a8e4409c3ee2c5ffe12f1395a11b095 WatchSource:0}: Error finding container b480b38026dedab69386e1826af2b1642a8e4409c3ee2c5ffe12f1395a11b095: Status 404 returned error can't find the container with id b480b38026dedab69386e1826af2b1642a8e4409c3ee2c5ffe12f1395a11b095
Jul 13 21:00:10.715989 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.715971    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:00:10.877811 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:10.877775    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z" event={"ID":"c0156185-4b50-4b11-9332-fd09471fa634","Type":"ContainerStarted","Data":"b480b38026dedab69386e1826af2b1642a8e4409c3ee2c5ffe12f1395a11b095"}
Jul 13 21:00:13.376684 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.376655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 21:00:13.378751 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.378731    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4035ca-3984-435e-8387-217677bcc954-metrics-certs\") pod \"network-metrics-daemon-grcrp\" (UID: \"de4035ca-3984-435e-8387-217677bcc954\") " pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 21:00:13.405878 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.405857    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-plnw6\""
Jul 13 21:00:13.414031 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.414012    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-grcrp"
Jul 13 21:00:13.564016 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.563990    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-grcrp"]
Jul 13 21:00:13.566191 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:00:13.566163    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4035ca_3984_435e_8387_217677bcc954.slice/crio-b26e0bf90bc84f6447c77fe13b652e84fe1ba6212a31433ba93891b09905ea98 WatchSource:0}: Error finding container b26e0bf90bc84f6447c77fe13b652e84fe1ba6212a31433ba93891b09905ea98: Status 404 returned error can't find the container with id b26e0bf90bc84f6447c77fe13b652e84fe1ba6212a31433ba93891b09905ea98
Jul 13 21:00:13.894904 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.894866    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z" event={"ID":"c0156185-4b50-4b11-9332-fd09471fa634","Type":"ContainerStarted","Data":"8bdd2c1299d55cd9980f3dc5cdf3f5bc2b23e0dc62322efaaedc4e34be534154"}
Jul 13 21:00:13.895083 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.894928    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z" event={"ID":"c0156185-4b50-4b11-9332-fd09471fa634","Type":"ContainerStarted","Data":"0750322ce0a2d42e423621897b85f19a595c69f18cd3903505266e4e6bc6473e"}
Jul 13 21:00:13.895083 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.894949    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z" event={"ID":"c0156185-4b50-4b11-9332-fd09471fa634","Type":"ContainerStarted","Data":"6743952aabbfd7c981fc6778acf1fcceea5ee91c08fff668884bfdcd47a783d7"}
Jul 13 21:00:13.896073 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.896047    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-grcrp" event={"ID":"de4035ca-3984-435e-8387-217677bcc954","Type":"ContainerStarted","Data":"b26e0bf90bc84f6447c77fe13b652e84fe1ba6212a31433ba93891b09905ea98"}
Jul 13 21:00:13.918500 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:13.918457    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5696968dc5-nxh7z" podStartSLOduration=1.286410111 podStartE2EDuration="3.918442184s" totalImagesPullingTime="2.632032073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:13.91630626 +0000 UTC m=+374.854510571" watchObservedRunningTime="2026-07-13 21:00:13.918442184 +0000 UTC m=+374.856646494"
Jul 13 21:00:14.824418 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.824321    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-795d45ddc-lxjvm"]
Jul 13 21:00:14.828397 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.828111    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:14.839862 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.839798    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795d45ddc-lxjvm"]
Jul 13 21:00:14.993279 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.993247    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-service-ca\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:14.993481 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.993397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-oauth-config\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:14.993598 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.993540    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-oauth-serving-cert\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:14.993598 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.993592    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcpmc\" (UniqueName: \"kubernetes.io/projected/636682fc-1398-4c8f-bea9-4d0d860e3c7b-kube-api-access-vcpmc\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:14.993787 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.993763    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-serving-cert\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:14.993973 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.993950    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-config\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:14.994088 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:14.993982    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-trusted-ca-bundle\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095064 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.094982    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-service-ca\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095064 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.095036    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-oauth-config\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095268 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.095071    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-oauth-serving-cert\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095268 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.095097    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcpmc\" (UniqueName: \"kubernetes.io/projected/636682fc-1398-4c8f-bea9-4d0d860e3c7b-kube-api-access-vcpmc\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095268 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.095133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-serving-cert\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095440 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.095302    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-config\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095440 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.095357    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-trusted-ca-bundle\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.095874 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.095853    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-service-ca\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.096097 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.096072    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-config\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.096293 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.096273    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-trusted-ca-bundle\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.096417 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.096328    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-oauth-serving-cert\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.097929 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.097911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-oauth-config\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.098006 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.097951    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-serving-cert\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.103823 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.103798    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcpmc\" (UniqueName: \"kubernetes.io/projected/636682fc-1398-4c8f-bea9-4d0d860e3c7b-kube-api-access-vcpmc\") pod \"console-795d45ddc-lxjvm\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") " pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.142669 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.142641    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:15.306172 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.306078    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795d45ddc-lxjvm"]
Jul 13 21:00:15.308545 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:00:15.308515    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636682fc_1398_4c8f_bea9_4d0d860e3c7b.slice/crio-6f83b564075771335b9207fd121648855f6a5c5bd045c03c4a4de34a3d0efa56 WatchSource:0}: Error finding container 6f83b564075771335b9207fd121648855f6a5c5bd045c03c4a4de34a3d0efa56: Status 404 returned error can't find the container with id 6f83b564075771335b9207fd121648855f6a5c5bd045c03c4a4de34a3d0efa56
Jul 13 21:00:15.907577 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.907532    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-grcrp" event={"ID":"de4035ca-3984-435e-8387-217677bcc954","Type":"ContainerStarted","Data":"29a190abc37f24cabbb6718b60ce3cf6909418f0d5f30f46395528724050287e"}
Jul 13 21:00:15.908027 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.907595    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-grcrp" event={"ID":"de4035ca-3984-435e-8387-217677bcc954","Type":"ContainerStarted","Data":"570976289eb67029437ffb1e7ac75ae2eca44eaa4fe25c0400fdb44e79735ab6"}
Jul 13 21:00:15.909010 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.908989    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795d45ddc-lxjvm" event={"ID":"636682fc-1398-4c8f-bea9-4d0d860e3c7b","Type":"ContainerStarted","Data":"14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745"}
Jul 13 21:00:15.909090 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.909023    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795d45ddc-lxjvm" event={"ID":"636682fc-1398-4c8f-bea9-4d0d860e3c7b","Type":"ContainerStarted","Data":"6f83b564075771335b9207fd121648855f6a5c5bd045c03c4a4de34a3d0efa56"}
Jul 13 21:00:15.924090 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:15.924051    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-grcrp" podStartSLOduration=375.316876146 podStartE2EDuration="6m16.924040008s" totalImagesPullingTime="1.607163862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:15.922769361 +0000 UTC m=+376.860973673" watchObservedRunningTime="2026-07-13 21:00:15.924040008 +0000 UTC m=+376.862244318"
Jul 13 21:00:25.143151 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:25.143117    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:25.143567 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:25.143174    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:25.147823 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:25.147800    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:25.170947 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:25.170903    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-795d45ddc-lxjvm" podStartSLOduration=11.1708907 podStartE2EDuration="11.1708907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:15.941989611 +0000 UTC m=+376.880193922" watchObservedRunningTime="2026-07-13 21:00:25.1708907 +0000 UTC m=+386.109095011"
Jul 13 21:00:25.954145 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:25.954118    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:00:25.999201 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:25.999159    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5c9677f5-tx7lc"]
Jul 13 21:00:51.023573 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.023514    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c5c9677f5-tx7lc" podUID="14221df1-7cd4-4636-a2b9-73488739fef1" containerName="console" containerID="cri-o://e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5" gracePeriod=15
Jul 13 21:00:51.269952 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.269929    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 21:00:51.379087 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379065    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-console-config\") pod \"14221df1-7cd4-4636-a2b9-73488739fef1\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") "
Jul 13 21:00:51.379234 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379095    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-oauth-config\") pod \"14221df1-7cd4-4636-a2b9-73488739fef1\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") "
Jul 13 21:00:51.379234 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379118    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-trusted-ca-bundle\") pod \"14221df1-7cd4-4636-a2b9-73488739fef1\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") "
Jul 13 21:00:51.379234 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379142    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-service-ca\") pod \"14221df1-7cd4-4636-a2b9-73488739fef1\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") "
Jul 13 21:00:51.379234 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379168    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-serving-cert\") pod \"14221df1-7cd4-4636-a2b9-73488739fef1\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") "
Jul 13 21:00:51.379234 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379214    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r74ls\" (UniqueName: \"kubernetes.io/projected/14221df1-7cd4-4636-a2b9-73488739fef1-kube-api-access-r74ls\") pod \"14221df1-7cd4-4636-a2b9-73488739fef1\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") "
Jul 13 21:00:51.379502 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379265    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-oauth-serving-cert\") pod \"14221df1-7cd4-4636-a2b9-73488739fef1\" (UID: \"14221df1-7cd4-4636-a2b9-73488739fef1\") "
Jul 13 21:00:51.379560 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379490    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-console-config" (OuterVolumeSpecName: "console-config") pod "14221df1-7cd4-4636-a2b9-73488739fef1" (UID: "14221df1-7cd4-4636-a2b9-73488739fef1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:00:51.379719 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379700    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-console-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:51.379788 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379752    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-service-ca" (OuterVolumeSpecName: "service-ca") pod "14221df1-7cd4-4636-a2b9-73488739fef1" (UID: "14221df1-7cd4-4636-a2b9-73488739fef1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:00:51.379846 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379784    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "14221df1-7cd4-4636-a2b9-73488739fef1" (UID: "14221df1-7cd4-4636-a2b9-73488739fef1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:00:51.379846 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.379805    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "14221df1-7cd4-4636-a2b9-73488739fef1" (UID: "14221df1-7cd4-4636-a2b9-73488739fef1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:00:51.381193 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.381169    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "14221df1-7cd4-4636-a2b9-73488739fef1" (UID: "14221df1-7cd4-4636-a2b9-73488739fef1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:51.381663 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.381638    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "14221df1-7cd4-4636-a2b9-73488739fef1" (UID: "14221df1-7cd4-4636-a2b9-73488739fef1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:00:51.381772 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.381750    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14221df1-7cd4-4636-a2b9-73488739fef1-kube-api-access-r74ls" (OuterVolumeSpecName: "kube-api-access-r74ls") pod "14221df1-7cd4-4636-a2b9-73488739fef1" (UID: "14221df1-7cd4-4636-a2b9-73488739fef1"). InnerVolumeSpecName "kube-api-access-r74ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:00:51.480532 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.480505    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r74ls\" (UniqueName: \"kubernetes.io/projected/14221df1-7cd4-4636-a2b9-73488739fef1-kube-api-access-r74ls\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:51.480532 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.480528    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-oauth-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:51.480532 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.480537    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-oauth-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:51.480721 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.480547    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-trusted-ca-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:51.480721 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.480555    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14221df1-7cd4-4636-a2b9-73488739fef1-service-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:51.480721 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:51.480563    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14221df1-7cd4-4636-a2b9-73488739fef1-console-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:00:52.053173 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.053140    2556 generic.go:356] "Generic (PLEG): container finished" podID="14221df1-7cd4-4636-a2b9-73488739fef1" containerID="e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5" exitCode=0
Jul 13 21:00:52.053640 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.053186    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9677f5-tx7lc"
Jul 13 21:00:52.053640 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.053196    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9677f5-tx7lc" event={"ID":"14221df1-7cd4-4636-a2b9-73488739fef1","Type":"ContainerDied","Data":"e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5"}
Jul 13 21:00:52.053640 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.053236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9677f5-tx7lc" event={"ID":"14221df1-7cd4-4636-a2b9-73488739fef1","Type":"ContainerDied","Data":"8c57bc71de33c5ee00252c62307ce3d78b66803819e8310dbdc7a9ff139e6e15"}
Jul 13 21:00:52.053640 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.053256    2556 scope.go:117] "RemoveContainer" containerID="e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5"
Jul 13 21:00:52.061713 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.061693    2556 scope.go:117] "RemoveContainer" containerID="e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5"
Jul 13 21:00:52.061978 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:00:52.061960    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5\": container with ID starting with e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5 not found: ID does not exist" containerID="e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5"
Jul 13 21:00:52.062032 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.061988    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5"} err="failed to get container status \"e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5\": rpc error: code = NotFound desc = could not find container \"e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5\": container with ID starting with e5c1a8eb606c8a11a7c2fd932ee6fe955db37e1fdbbcf5781e2d79e168e688b5 not found: ID does not exist"
Jul 13 21:00:52.069677 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.069655    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5c9677f5-tx7lc"]
Jul 13 21:00:52.077307 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:52.077287    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5c9677f5-tx7lc"]
Jul 13 21:00:53.601636 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:00:53.601607    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14221df1-7cd4-4636-a2b9-73488739fef1" path="/var/lib/kubelet/pods/14221df1-7cd4-4636-a2b9-73488739fef1/volumes"
Jul 13 21:01:30.466529 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.466450    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db5d69799-85nfw"]
Jul 13 21:01:30.466956 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.466808    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14221df1-7cd4-4636-a2b9-73488739fef1" containerName="console"
Jul 13 21:01:30.466956 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.466820    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="14221df1-7cd4-4636-a2b9-73488739fef1" containerName="console"
Jul 13 21:01:30.466956 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.466898    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="14221df1-7cd4-4636-a2b9-73488739fef1" containerName="console"
Jul 13 21:01:30.469811 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.469791    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.480856 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.480832    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db5d69799-85nfw"]
Jul 13 21:01:30.602477 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.602449    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-console-config\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.602646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.602488    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-oauth-serving-cert\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.602646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.602506    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv72m\" (UniqueName: \"kubernetes.io/projected/9623e620-2c82-4355-868c-bdf48691a417-kube-api-access-nv72m\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.602646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.602537    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-oauth-config\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.602646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.602602    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-serving-cert\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.602646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.602636    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-service-ca\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.602812 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.602656    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-trusted-ca-bundle\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.703641 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.703610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-serving-cert\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.703786 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.703650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-service-ca\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.703786 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.703670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-trusted-ca-bundle\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.703786 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.703712    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-console-config\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.703786 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.703749    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-oauth-serving-cert\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.703786 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.703771    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv72m\" (UniqueName: \"kubernetes.io/projected/9623e620-2c82-4355-868c-bdf48691a417-kube-api-access-nv72m\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.704040 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.703811    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-oauth-config\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.704519 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.704495    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-oauth-serving-cert\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.704519 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.704513    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-service-ca\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.704657 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.704600    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-trusted-ca-bundle\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.704657 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.704600    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-console-config\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.706192 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.706171    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-serving-cert\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.706192 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.706184    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-oauth-config\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.712129 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.712112    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv72m\" (UniqueName: \"kubernetes.io/projected/9623e620-2c82-4355-868c-bdf48691a417-kube-api-access-nv72m\") pod \"console-5db5d69799-85nfw\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") " pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.778882 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.778832    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:30.911567 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:30.911537    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db5d69799-85nfw"]
Jul 13 21:01:30.915032 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:01:30.915009    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9623e620_2c82_4355_868c_bdf48691a417.slice/crio-5c536c31149e7892f48cf92ecb33ee8fb8fbcdb38eb1c9a2913a6180ef4d381b WatchSource:0}: Error finding container 5c536c31149e7892f48cf92ecb33ee8fb8fbcdb38eb1c9a2913a6180ef4d381b: Status 404 returned error can't find the container with id 5c536c31149e7892f48cf92ecb33ee8fb8fbcdb38eb1c9a2913a6180ef4d381b
Jul 13 21:01:31.219133 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:31.219094    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db5d69799-85nfw" event={"ID":"9623e620-2c82-4355-868c-bdf48691a417","Type":"ContainerStarted","Data":"de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100"}
Jul 13 21:01:31.219296 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:31.219156    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db5d69799-85nfw" event={"ID":"9623e620-2c82-4355-868c-bdf48691a417","Type":"ContainerStarted","Data":"5c536c31149e7892f48cf92ecb33ee8fb8fbcdb38eb1c9a2913a6180ef4d381b"}
Jul 13 21:01:31.244672 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:31.244618    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db5d69799-85nfw" podStartSLOduration=1.244602564 podStartE2EDuration="1.244602564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:01:31.243090506 +0000 UTC m=+452.181294837" watchObservedRunningTime="2026-07-13 21:01:31.244602564 +0000 UTC m=+452.182806875"
Jul 13 21:01:40.779397 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:40.779364    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:40.779727 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:40.779421    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:40.783788 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:40.783765    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:41.263789 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:41.263761    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:01:41.335694 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:01:41.335659    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-795d45ddc-lxjvm"]
Jul 13 21:02:06.360478 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.360425    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-795d45ddc-lxjvm" podUID="636682fc-1398-4c8f-bea9-4d0d860e3c7b" containerName="console" containerID="cri-o://14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745" gracePeriod=15
Jul 13 21:02:06.595889 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.595866    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:02:06.701915 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.701846    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-serving-cert\") pod \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") "
Jul 13 21:02:06.701915 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.701880    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-oauth-serving-cert\") pod \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") "
Jul 13 21:02:06.701915 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.701912    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-trusted-ca-bundle\") pod \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") "
Jul 13 21:02:06.702113 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.701966    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-service-ca\") pod \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") "
Jul 13 21:02:06.702113 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702005    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcpmc\" (UniqueName: \"kubernetes.io/projected/636682fc-1398-4c8f-bea9-4d0d860e3c7b-kube-api-access-vcpmc\") pod \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") "
Jul 13 21:02:06.702113 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702069    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-oauth-config\") pod \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") "
Jul 13 21:02:06.702294 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702123    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-config\") pod \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\" (UID: \"636682fc-1398-4c8f-bea9-4d0d860e3c7b\") "
Jul 13 21:02:06.702378 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702323    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "636682fc-1398-4c8f-bea9-4d0d860e3c7b" (UID: "636682fc-1398-4c8f-bea9-4d0d860e3c7b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:02:06.702450 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702430    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-service-ca" (OuterVolumeSpecName: "service-ca") pod "636682fc-1398-4c8f-bea9-4d0d860e3c7b" (UID: "636682fc-1398-4c8f-bea9-4d0d860e3c7b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:02:06.702502 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702444    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-oauth-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:02:06.702502 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702437    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "636682fc-1398-4c8f-bea9-4d0d860e3c7b" (UID: "636682fc-1398-4c8f-bea9-4d0d860e3c7b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:02:06.702751 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.702728    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-config" (OuterVolumeSpecName: "console-config") pod "636682fc-1398-4c8f-bea9-4d0d860e3c7b" (UID: "636682fc-1398-4c8f-bea9-4d0d860e3c7b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:02:06.704139 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.704112    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/636682fc-1398-4c8f-bea9-4d0d860e3c7b-kube-api-access-vcpmc" (OuterVolumeSpecName: "kube-api-access-vcpmc") pod "636682fc-1398-4c8f-bea9-4d0d860e3c7b" (UID: "636682fc-1398-4c8f-bea9-4d0d860e3c7b"). InnerVolumeSpecName "kube-api-access-vcpmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:02:06.704232 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.704136    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "636682fc-1398-4c8f-bea9-4d0d860e3c7b" (UID: "636682fc-1398-4c8f-bea9-4d0d860e3c7b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:02:06.704232 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.704157    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "636682fc-1398-4c8f-bea9-4d0d860e3c7b" (UID: "636682fc-1398-4c8f-bea9-4d0d860e3c7b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:02:06.803705 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.803681    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:02:06.803705 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.803703    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-trusted-ca-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:02:06.803887 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.803714    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-service-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:02:06.803887 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.803722    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcpmc\" (UniqueName: \"kubernetes.io/projected/636682fc-1398-4c8f-bea9-4d0d860e3c7b-kube-api-access-vcpmc\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:02:06.803887 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.803732    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-oauth-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:02:06.803887 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:06.803740    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636682fc-1398-4c8f-bea9-4d0d860e3c7b-console-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:02:07.366761 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.366726    2556 generic.go:356] "Generic (PLEG): container finished" podID="636682fc-1398-4c8f-bea9-4d0d860e3c7b" containerID="14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745" exitCode=0
Jul 13 21:02:07.367149 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.366772    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795d45ddc-lxjvm"
Jul 13 21:02:07.367149 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.366807    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795d45ddc-lxjvm" event={"ID":"636682fc-1398-4c8f-bea9-4d0d860e3c7b","Type":"ContainerDied","Data":"14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745"}
Jul 13 21:02:07.367149 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.366855    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795d45ddc-lxjvm" event={"ID":"636682fc-1398-4c8f-bea9-4d0d860e3c7b","Type":"ContainerDied","Data":"6f83b564075771335b9207fd121648855f6a5c5bd045c03c4a4de34a3d0efa56"}
Jul 13 21:02:07.367149 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.366874    2556 scope.go:117] "RemoveContainer" containerID="14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745"
Jul 13 21:02:07.375233 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.375218    2556 scope.go:117] "RemoveContainer" containerID="14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745"
Jul 13 21:02:07.375493 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:02:07.375475    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745\": container with ID starting with 14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745 not found: ID does not exist" containerID="14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745"
Jul 13 21:02:07.375545 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.375500    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745"} err="failed to get container status \"14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745\": rpc error: code = NotFound desc = could not find container \"14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745\": container with ID starting with 14b605a25006d9622a745eee274f2a6af28bb1bcc2087bef1d81834d5977a745 not found: ID does not exist"
Jul 13 21:02:07.386195 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.386174    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-795d45ddc-lxjvm"]
Jul 13 21:02:07.390047 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.390026    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-795d45ddc-lxjvm"]
Jul 13 21:02:07.602394 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:07.602367    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="636682fc-1398-4c8f-bea9-4d0d860e3c7b" path="/var/lib/kubelet/pods/636682fc-1398-4c8f-bea9-4d0d860e3c7b/volumes"
Jul 13 21:02:39.422466 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.422433    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"]
Jul 13 21:02:39.422907 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.422752    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="636682fc-1398-4c8f-bea9-4d0d860e3c7b" containerName="console"
Jul 13 21:02:39.422907 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.422764    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="636682fc-1398-4c8f-bea9-4d0d860e3c7b" containerName="console"
Jul 13 21:02:39.422907 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.422820    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="636682fc-1398-4c8f-bea9-4d0d860e3c7b" containerName="console"
Jul 13 21:02:39.425921 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.425906    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.428228 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.428204    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zlxgm\""
Jul 13 21:02:39.428351 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.428268    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 13 21:02:39.429385 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.429370    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 13 21:02:39.434329 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.434307    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"]
Jul 13 21:02:39.475490 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.475458    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.475604 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.475498    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfzx6\" (UniqueName: \"kubernetes.io/projected/be849b4a-25eb-47f1-9716-993490be6ed1-kube-api-access-sfzx6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.475604 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.475522    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.576854 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.576825    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.577003 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.576857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfzx6\" (UniqueName: \"kubernetes.io/projected/be849b4a-25eb-47f1-9716-993490be6ed1-kube-api-access-sfzx6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.577003 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.576879    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.577219 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.577199    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.577275 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.577231    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.586396 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.586370    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfzx6\" (UniqueName: \"kubernetes.io/projected/be849b4a-25eb-47f1-9716-993490be6ed1-kube-api-access-sfzx6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.735296 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.735220    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:02:39.864293 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:39.864268    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"]
Jul 13 21:02:39.866179 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:02:39.866152    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe849b4a_25eb_47f1_9716_993490be6ed1.slice/crio-ed2a49af1f2e83b596b4bdd012c750ba815ffbd7f58ed94f1d0531f90b860a36 WatchSource:0}: Error finding container ed2a49af1f2e83b596b4bdd012c750ba815ffbd7f58ed94f1d0531f90b860a36: Status 404 returned error can't find the container with id ed2a49af1f2e83b596b4bdd012c750ba815ffbd7f58ed94f1d0531f90b860a36
Jul 13 21:02:40.501385 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:40.501355    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch" event={"ID":"be849b4a-25eb-47f1-9716-993490be6ed1","Type":"ContainerStarted","Data":"ed2a49af1f2e83b596b4bdd012c750ba815ffbd7f58ed94f1d0531f90b860a36"}
Jul 13 21:02:47.541493 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:47.541460    2556 generic.go:356] "Generic (PLEG): container finished" podID="be849b4a-25eb-47f1-9716-993490be6ed1" containerID="c84a44155459ffb0b2d699fdac46943980817c8ce7af13a26f3f9a6057410186" exitCode=0
Jul 13 21:02:47.541858 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:47.541548    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch" event={"ID":"be849b4a-25eb-47f1-9716-993490be6ed1","Type":"ContainerDied","Data":"c84a44155459ffb0b2d699fdac46943980817c8ce7af13a26f3f9a6057410186"}
Jul 13 21:02:51.561105 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:51.561071    2556 generic.go:356] "Generic (PLEG): container finished" podID="be849b4a-25eb-47f1-9716-993490be6ed1" containerID="c309de7f06c00bb4aa02135f0b31224a96fd00110e64543bf2acb7b8965850db" exitCode=0
Jul 13 21:02:51.561486 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:02:51.561121    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch" event={"ID":"be849b4a-25eb-47f1-9716-993490be6ed1","Type":"ContainerDied","Data":"c309de7f06c00bb4aa02135f0b31224a96fd00110e64543bf2acb7b8965850db"}
Jul 13 21:03:00.613410 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:00.613378    2556 generic.go:356] "Generic (PLEG): container finished" podID="be849b4a-25eb-47f1-9716-993490be6ed1" containerID="12f6ebd96da8243d691efbb118b9b2f53c573ab192fd3ee7065e955e827414e5" exitCode=0
Jul 13 21:03:00.613410 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:00.613413    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch" event={"ID":"be849b4a-25eb-47f1-9716-993490be6ed1","Type":"ContainerDied","Data":"12f6ebd96da8243d691efbb118b9b2f53c573ab192fd3ee7065e955e827414e5"}
Jul 13 21:03:01.745035 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.745008    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:03:01.868393 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.868290    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfzx6\" (UniqueName: \"kubernetes.io/projected/be849b4a-25eb-47f1-9716-993490be6ed1-kube-api-access-sfzx6\") pod \"be849b4a-25eb-47f1-9716-993490be6ed1\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") "
Jul 13 21:03:01.868393 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.868390    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-bundle\") pod \"be849b4a-25eb-47f1-9716-993490be6ed1\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") "
Jul 13 21:03:01.868568 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.868484    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-util\") pod \"be849b4a-25eb-47f1-9716-993490be6ed1\" (UID: \"be849b4a-25eb-47f1-9716-993490be6ed1\") "
Jul 13 21:03:01.869042 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.869013    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-bundle" (OuterVolumeSpecName: "bundle") pod "be849b4a-25eb-47f1-9716-993490be6ed1" (UID: "be849b4a-25eb-47f1-9716-993490be6ed1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:03:01.870573 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.870554    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be849b4a-25eb-47f1-9716-993490be6ed1-kube-api-access-sfzx6" (OuterVolumeSpecName: "kube-api-access-sfzx6") pod "be849b4a-25eb-47f1-9716-993490be6ed1" (UID: "be849b4a-25eb-47f1-9716-993490be6ed1"). InnerVolumeSpecName "kube-api-access-sfzx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:03:01.872931 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.872909    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-util" (OuterVolumeSpecName: "util") pod "be849b4a-25eb-47f1-9716-993490be6ed1" (UID: "be849b4a-25eb-47f1-9716-993490be6ed1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:03:01.969788 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.969748    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-util\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:01.969788 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.969779    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfzx6\" (UniqueName: \"kubernetes.io/projected/be849b4a-25eb-47f1-9716-993490be6ed1-kube-api-access-sfzx6\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:01.969788 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:01.969788    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be849b4a-25eb-47f1-9716-993490be6ed1-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:02.623061 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:02.623029    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch" event={"ID":"be849b4a-25eb-47f1-9716-993490be6ed1","Type":"ContainerDied","Data":"ed2a49af1f2e83b596b4bdd012c750ba815ffbd7f58ed94f1d0531f90b860a36"}
Jul 13 21:03:02.623226 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:02.623072    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed2a49af1f2e83b596b4bdd012c750ba815ffbd7f58ed94f1d0531f90b860a36"
Jul 13 21:03:02.623226 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:02.623072    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kljch"
Jul 13 21:03:07.734290 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734258    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"]
Jul 13 21:03:07.734662 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734580    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be849b4a-25eb-47f1-9716-993490be6ed1" containerName="util"
Jul 13 21:03:07.734662 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734591    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="be849b4a-25eb-47f1-9716-993490be6ed1" containerName="util"
Jul 13 21:03:07.734662 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734598    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be849b4a-25eb-47f1-9716-993490be6ed1" containerName="pull"
Jul 13 21:03:07.734662 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734604    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="be849b4a-25eb-47f1-9716-993490be6ed1" containerName="pull"
Jul 13 21:03:07.734662 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734612    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be849b4a-25eb-47f1-9716-993490be6ed1" containerName="extract"
Jul 13 21:03:07.734662 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734618    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="be849b4a-25eb-47f1-9716-993490be6ed1" containerName="extract"
Jul 13 21:03:07.734888 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.734677    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="be849b4a-25eb-47f1-9716-993490be6ed1" containerName="extract"
Jul 13 21:03:07.737829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.737812    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:07.740841 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.740824    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2sqp5\""
Jul 13 21:03:07.740922 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.740896    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:03:07.740969 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.740957    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 13 21:03:07.764088 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.764061    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"]
Jul 13 21:03:07.918389 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.918331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsl9x\" (UniqueName: \"kubernetes.io/projected/c147d906-01c9-4ab2-ad03-6755508e44a7-kube-api-access-zsl9x\") pod \"cert-manager-operator-controller-manager-55c444bf86-s8stv\" (UID: \"c147d906-01c9-4ab2-ad03-6755508e44a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:07.918389 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:07.918392    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c147d906-01c9-4ab2-ad03-6755508e44a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s8stv\" (UID: \"c147d906-01c9-4ab2-ad03-6755508e44a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:08.019226 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:08.019148    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsl9x\" (UniqueName: \"kubernetes.io/projected/c147d906-01c9-4ab2-ad03-6755508e44a7-kube-api-access-zsl9x\") pod \"cert-manager-operator-controller-manager-55c444bf86-s8stv\" (UID: \"c147d906-01c9-4ab2-ad03-6755508e44a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:08.019226 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:08.019180    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c147d906-01c9-4ab2-ad03-6755508e44a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s8stv\" (UID: \"c147d906-01c9-4ab2-ad03-6755508e44a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:08.019532 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:08.019517    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c147d906-01c9-4ab2-ad03-6755508e44a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s8stv\" (UID: \"c147d906-01c9-4ab2-ad03-6755508e44a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:08.037844 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:08.037821    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsl9x\" (UniqueName: \"kubernetes.io/projected/c147d906-01c9-4ab2-ad03-6755508e44a7-kube-api-access-zsl9x\") pod \"cert-manager-operator-controller-manager-55c444bf86-s8stv\" (UID: \"c147d906-01c9-4ab2-ad03-6755508e44a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:08.046659 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:08.046641    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"
Jul 13 21:03:08.177055 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:08.177026    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv"]
Jul 13 21:03:08.180275 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:03:08.180241    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc147d906_01c9_4ab2_ad03_6755508e44a7.slice/crio-1022399421be24cf2da3b195fdb2f08343ae660f416b1660470210743d7cc312 WatchSource:0}: Error finding container 1022399421be24cf2da3b195fdb2f08343ae660f416b1660470210743d7cc312: Status 404 returned error can't find the container with id 1022399421be24cf2da3b195fdb2f08343ae660f416b1660470210743d7cc312
Jul 13 21:03:08.648247 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:08.648220    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv" event={"ID":"c147d906-01c9-4ab2-ad03-6755508e44a7","Type":"ContainerStarted","Data":"1022399421be24cf2da3b195fdb2f08343ae660f416b1660470210743d7cc312"}
Jul 13 21:03:10.659412 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:10.659373    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv" event={"ID":"c147d906-01c9-4ab2-ad03-6755508e44a7","Type":"ContainerStarted","Data":"82c240b610fcd90575fb06f1770ec65f78fe3045b6a3e1d53143fc109bc6a4a9"}
Jul 13 21:03:12.185279 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.185228    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s8stv" podStartSLOduration=3.612591131 podStartE2EDuration="5.185212483s" totalImagesPullingTime="1.572621352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:03:10.711467517 +0000 UTC m=+551.649671857" watchObservedRunningTime="2026-07-13 21:03:12.185212483 +0000 UTC m=+553.123416835"
Jul 13 21:03:12.186277 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.186259    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"]
Jul 13 21:03:12.188670 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.188657    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.197580 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.197565    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 13 21:03:12.197663 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.197568    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 13 21:03:12.198614 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.198595    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zlxgm\""
Jul 13 21:03:12.221422 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.221399    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"]
Jul 13 21:03:12.360033 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.360002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w9n8\" (UniqueName: \"kubernetes.io/projected/e498bd59-29b8-4075-8593-b7b3c38f8a52-kube-api-access-5w9n8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.360191 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.360075    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.360191 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.360096    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.461384 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.461293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.461384 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.461328    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.461571 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.461388    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w9n8\" (UniqueName: \"kubernetes.io/projected/e498bd59-29b8-4075-8593-b7b3c38f8a52-kube-api-access-5w9n8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.461672 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.461643    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.461711 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.461673    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.475493 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.475464    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w9n8\" (UniqueName: \"kubernetes.io/projected/e498bd59-29b8-4075-8593-b7b3c38f8a52-kube-api-access-5w9n8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.497312 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.497292    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:12.626840 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.626814    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"]
Jul 13 21:03:12.628333 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:03:12.628309    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode498bd59_29b8_4075_8593_b7b3c38f8a52.slice/crio-78fea0ba46d91816d8ac4662afddbe9b1ad31f053be1b9ab7b10ac54ebd1ea7a WatchSource:0}: Error finding container 78fea0ba46d91816d8ac4662afddbe9b1ad31f053be1b9ab7b10ac54ebd1ea7a: Status 404 returned error can't find the container with id 78fea0ba46d91816d8ac4662afddbe9b1ad31f053be1b9ab7b10ac54ebd1ea7a
Jul 13 21:03:12.668715 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:12.668672    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx" event={"ID":"e498bd59-29b8-4075-8593-b7b3c38f8a52","Type":"ContainerStarted","Data":"78fea0ba46d91816d8ac4662afddbe9b1ad31f053be1b9ab7b10ac54ebd1ea7a"}
Jul 13 21:03:13.674693 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:13.674660    2556 generic.go:356] "Generic (PLEG): container finished" podID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerID="ddcb772fbc709e63538e398666ffa9161326b0d3b2e01ac66fe5080947233ec7" exitCode=0
Jul 13 21:03:13.675105 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:13.674742    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx" event={"ID":"e498bd59-29b8-4075-8593-b7b3c38f8a52","Type":"ContainerDied","Data":"ddcb772fbc709e63538e398666ffa9161326b0d3b2e01ac66fe5080947233ec7"}
Jul 13 21:03:14.785029 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.784997    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"]
Jul 13 21:03:14.787673 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.787653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:14.805846 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.805820    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 13 21:03:14.806041 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.806024    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dwqbb\""
Jul 13 21:03:14.806156 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.806040    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 13 21:03:14.808456 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.808426    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"]
Jul 13 21:03:14.883141 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.883104    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd986b2b-18e2-42d8-87bc-2133c1c98799-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q8f66\" (UID: \"cd986b2b-18e2-42d8-87bc-2133c1c98799\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:14.883300 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.883162    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfgsm\" (UniqueName: \"kubernetes.io/projected/cd986b2b-18e2-42d8-87bc-2133c1c98799-kube-api-access-zfgsm\") pod \"cert-manager-cainjector-559cdd8c87-q8f66\" (UID: \"cd986b2b-18e2-42d8-87bc-2133c1c98799\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:14.984639 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.984604    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd986b2b-18e2-42d8-87bc-2133c1c98799-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q8f66\" (UID: \"cd986b2b-18e2-42d8-87bc-2133c1c98799\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:14.984830 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.984667    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfgsm\" (UniqueName: \"kubernetes.io/projected/cd986b2b-18e2-42d8-87bc-2133c1c98799-kube-api-access-zfgsm\") pod \"cert-manager-cainjector-559cdd8c87-q8f66\" (UID: \"cd986b2b-18e2-42d8-87bc-2133c1c98799\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:14.995067 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.995042    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd986b2b-18e2-42d8-87bc-2133c1c98799-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-q8f66\" (UID: \"cd986b2b-18e2-42d8-87bc-2133c1c98799\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:14.995204 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:14.995165    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfgsm\" (UniqueName: \"kubernetes.io/projected/cd986b2b-18e2-42d8-87bc-2133c1c98799-kube-api-access-zfgsm\") pod \"cert-manager-cainjector-559cdd8c87-q8f66\" (UID: \"cd986b2b-18e2-42d8-87bc-2133c1c98799\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:15.108044 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:15.107917    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"
Jul 13 21:03:15.914463 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:15.914268    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-q8f66"]
Jul 13 21:03:15.916996 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:03:15.916973    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd986b2b_18e2_42d8_87bc_2133c1c98799.slice/crio-22178daa76b96ee8821fff054bab7ef1c791697563d2c5b23245047e546fc234 WatchSource:0}: Error finding container 22178daa76b96ee8821fff054bab7ef1c791697563d2c5b23245047e546fc234: Status 404 returned error can't find the container with id 22178daa76b96ee8821fff054bab7ef1c791697563d2c5b23245047e546fc234
Jul 13 21:03:16.692727 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:16.692691    2556 generic.go:356] "Generic (PLEG): container finished" podID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerID="78292e2765af6e834cf8ac79548fc88198c8e37573a50d3c535a3886af813410" exitCode=0
Jul 13 21:03:16.692914 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:16.692768    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx" event={"ID":"e498bd59-29b8-4075-8593-b7b3c38f8a52","Type":"ContainerDied","Data":"78292e2765af6e834cf8ac79548fc88198c8e37573a50d3c535a3886af813410"}
Jul 13 21:03:16.693921 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:16.693902    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66" event={"ID":"cd986b2b-18e2-42d8-87bc-2133c1c98799","Type":"ContainerStarted","Data":"22178daa76b96ee8821fff054bab7ef1c791697563d2c5b23245047e546fc234"}
Jul 13 21:03:17.702358 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:17.702306    2556 generic.go:356] "Generic (PLEG): container finished" podID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerID="f655ec0b8fe0a53036e2706c174a6e26d6719c21f1aa523ce42eceb769cc04cc" exitCode=0
Jul 13 21:03:17.702777 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:17.702388    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx" event={"ID":"e498bd59-29b8-4075-8593-b7b3c38f8a52","Type":"ContainerDied","Data":"f655ec0b8fe0a53036e2706c174a6e26d6719c21f1aa523ce42eceb769cc04cc"}
Jul 13 21:03:18.856088 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:18.856061    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:18.922663 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:18.922633    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-util\") pod \"e498bd59-29b8-4075-8593-b7b3c38f8a52\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") "
Jul 13 21:03:18.922806 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:18.922730    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w9n8\" (UniqueName: \"kubernetes.io/projected/e498bd59-29b8-4075-8593-b7b3c38f8a52-kube-api-access-5w9n8\") pod \"e498bd59-29b8-4075-8593-b7b3c38f8a52\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") "
Jul 13 21:03:18.922806 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:18.922767    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-bundle\") pod \"e498bd59-29b8-4075-8593-b7b3c38f8a52\" (UID: \"e498bd59-29b8-4075-8593-b7b3c38f8a52\") "
Jul 13 21:03:18.923133 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:18.923107    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-bundle" (OuterVolumeSpecName: "bundle") pod "e498bd59-29b8-4075-8593-b7b3c38f8a52" (UID: "e498bd59-29b8-4075-8593-b7b3c38f8a52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:03:18.925008 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:18.924986    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e498bd59-29b8-4075-8593-b7b3c38f8a52-kube-api-access-5w9n8" (OuterVolumeSpecName: "kube-api-access-5w9n8") pod "e498bd59-29b8-4075-8593-b7b3c38f8a52" (UID: "e498bd59-29b8-4075-8593-b7b3c38f8a52"). InnerVolumeSpecName "kube-api-access-5w9n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:03:18.928525 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:18.928497    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-util" (OuterVolumeSpecName: "util") pod "e498bd59-29b8-4075-8593-b7b3c38f8a52" (UID: "e498bd59-29b8-4075-8593-b7b3c38f8a52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:03:19.023958 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.023870    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:19.023958 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.023904    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e498bd59-29b8-4075-8593-b7b3c38f8a52-util\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:19.023958 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.023921    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w9n8\" (UniqueName: \"kubernetes.io/projected/e498bd59-29b8-4075-8593-b7b3c38f8a52-kube-api-access-5w9n8\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:19.713699 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.713665    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66" event={"ID":"cd986b2b-18e2-42d8-87bc-2133c1c98799","Type":"ContainerStarted","Data":"9af6112676a3bf321b2496dc112331179e10c532995e1b74270e9d49357de31e"}
Jul 13 21:03:19.715334 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.715307    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx" event={"ID":"e498bd59-29b8-4075-8593-b7b3c38f8a52","Type":"ContainerDied","Data":"78fea0ba46d91816d8ac4662afddbe9b1ad31f053be1b9ab7b10ac54ebd1ea7a"}
Jul 13 21:03:19.715334 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.715325    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbbsvx"
Jul 13 21:03:19.715508 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.715374    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78fea0ba46d91816d8ac4662afddbe9b1ad31f053be1b9ab7b10ac54ebd1ea7a"
Jul 13 21:03:19.733174 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:19.733131    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-q8f66" podStartSLOduration=2.8637945240000002 podStartE2EDuration="5.733116791s" totalImagesPullingTime="2.869322267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:03:19.731561957 +0000 UTC m=+560.669766272" watchObservedRunningTime="2026-07-13 21:03:19.733116791 +0000 UTC m=+560.671321102"
Jul 13 21:03:26.003734 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.003703    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"]
Jul 13 21:03:26.004106 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.004079    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerName="extract"
Jul 13 21:03:26.004106 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.004094    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerName="extract"
Jul 13 21:03:26.004183 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.004113    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerName="pull"
Jul 13 21:03:26.004183 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.004119    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerName="pull"
Jul 13 21:03:26.004183 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.004133    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerName="util"
Jul 13 21:03:26.004183 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.004139    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerName="util"
Jul 13 21:03:26.004295 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.004188    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e498bd59-29b8-4075-8593-b7b3c38f8a52" containerName="extract"
Jul 13 21:03:26.006109 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.006094    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.009661 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.009642    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 13 21:03:26.010458 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.010442    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:03:26.010552 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.010447    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-q4n8d\""
Jul 13 21:03:26.025125 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.025099    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"]
Jul 13 21:03:26.078292 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.078265    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3020b304-ddbc-4aae-9769-40fbecd67cb4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m47nk\" (UID: \"3020b304-ddbc-4aae-9769-40fbecd67cb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.078457 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.078312    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7k2b\" (UniqueName: \"kubernetes.io/projected/3020b304-ddbc-4aae-9769-40fbecd67cb4-kube-api-access-z7k2b\") pod \"openshift-lws-operator-fd8ccff4c-m47nk\" (UID: \"3020b304-ddbc-4aae-9769-40fbecd67cb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.179361 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.179292    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3020b304-ddbc-4aae-9769-40fbecd67cb4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m47nk\" (UID: \"3020b304-ddbc-4aae-9769-40fbecd67cb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.179514 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.179401    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7k2b\" (UniqueName: \"kubernetes.io/projected/3020b304-ddbc-4aae-9769-40fbecd67cb4-kube-api-access-z7k2b\") pod \"openshift-lws-operator-fd8ccff4c-m47nk\" (UID: \"3020b304-ddbc-4aae-9769-40fbecd67cb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.179714 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.179697    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3020b304-ddbc-4aae-9769-40fbecd67cb4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m47nk\" (UID: \"3020b304-ddbc-4aae-9769-40fbecd67cb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.188727 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.188699    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7k2b\" (UniqueName: \"kubernetes.io/projected/3020b304-ddbc-4aae-9769-40fbecd67cb4-kube-api-access-z7k2b\") pod \"openshift-lws-operator-fd8ccff4c-m47nk\" (UID: \"3020b304-ddbc-4aae-9769-40fbecd67cb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.315461 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.315382    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"
Jul 13 21:03:26.453578 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.453556    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk"]
Jul 13 21:03:26.455650 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:03:26.455620    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3020b304_ddbc_4aae_9769_40fbecd67cb4.slice/crio-ddbbf8dad2e68e877970b26c9cbdf73fc29c8dc57e18d5e1518eec8bc94e5ded WatchSource:0}: Error finding container ddbbf8dad2e68e877970b26c9cbdf73fc29c8dc57e18d5e1518eec8bc94e5ded: Status 404 returned error can't find the container with id ddbbf8dad2e68e877970b26c9cbdf73fc29c8dc57e18d5e1518eec8bc94e5ded
Jul 13 21:03:26.755522 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:26.755482    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk" event={"ID":"3020b304-ddbc-4aae-9769-40fbecd67cb4","Type":"ContainerStarted","Data":"ddbbf8dad2e68e877970b26c9cbdf73fc29c8dc57e18d5e1518eec8bc94e5ded"}
Jul 13 21:03:29.771641 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:29.771603    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk" event={"ID":"3020b304-ddbc-4aae-9769-40fbecd67cb4","Type":"ContainerStarted","Data":"bff61acf5777c1c6b334b882704d5f7732d57aeee44d7b652d52dc52bf535f81"}
Jul 13 21:03:29.792826 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:29.792781    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m47nk" podStartSLOduration=2.062484942 podStartE2EDuration="4.792765013s" totalImagesPullingTime="2.730280071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:03:29.790985454 +0000 UTC m=+570.729189784" watchObservedRunningTime="2026-07-13 21:03:29.792765013 +0000 UTC m=+570.730969323"
Jul 13 21:03:32.474910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.474875    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"]
Jul 13 21:03:32.488527 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.488501    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"]
Jul 13 21:03:32.488662 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.488604    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.491785 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.491761    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zlxgm\""
Jul 13 21:03:32.492025 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.492011    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 13 21:03:32.493140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.493116    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 13 21:03:32.531000 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.530968    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.531000 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.531003    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4fbc\" (UniqueName: \"kubernetes.io/projected/0b969012-9179-4f98-bb13-0d5f146fdfaf-kube-api-access-v4fbc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.531180 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.531153    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.632506 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.632474    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.632506 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.632508    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.632722 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.632527    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4fbc\" (UniqueName: \"kubernetes.io/projected/0b969012-9179-4f98-bb13-0d5f146fdfaf-kube-api-access-v4fbc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.632890 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.632870    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.632927 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.632891    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.641418 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.641393    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4fbc\" (UniqueName: \"kubernetes.io/projected/0b969012-9179-4f98-bb13-0d5f146fdfaf-kube-api-access-v4fbc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.800302 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.800227    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:32.943348 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:32.943316    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"]
Jul 13 21:03:32.945613 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:03:32.945591    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b969012_9179_4f98_bb13_0d5f146fdfaf.slice/crio-d04c69116af9305102c4fdccd3b7e7064a89687b826b51e14d6d54e5a909c938 WatchSource:0}: Error finding container d04c69116af9305102c4fdccd3b7e7064a89687b826b51e14d6d54e5a909c938: Status 404 returned error can't find the container with id d04c69116af9305102c4fdccd3b7e7064a89687b826b51e14d6d54e5a909c938
Jul 13 21:03:33.791507 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:33.791474    2556 generic.go:356] "Generic (PLEG): container finished" podID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerID="ee73636e94890dd445857eeb141bba10f202223e1a3a2642c428cb92d64f3bea" exitCode=0
Jul 13 21:03:33.791872 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:33.791526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx" event={"ID":"0b969012-9179-4f98-bb13-0d5f146fdfaf","Type":"ContainerDied","Data":"ee73636e94890dd445857eeb141bba10f202223e1a3a2642c428cb92d64f3bea"}
Jul 13 21:03:33.791872 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:33.791571    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx" event={"ID":"0b969012-9179-4f98-bb13-0d5f146fdfaf","Type":"ContainerStarted","Data":"d04c69116af9305102c4fdccd3b7e7064a89687b826b51e14d6d54e5a909c938"}
Jul 13 21:03:35.802162 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:35.802124    2556 generic.go:356] "Generic (PLEG): container finished" podID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerID="c16e16276a79cf1b505a779592c54504210f64d104d6b17725cc3b5597c71c93" exitCode=0
Jul 13 21:03:35.802601 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:35.802210    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx" event={"ID":"0b969012-9179-4f98-bb13-0d5f146fdfaf","Type":"ContainerDied","Data":"c16e16276a79cf1b505a779592c54504210f64d104d6b17725cc3b5597c71c93"}
Jul 13 21:03:36.808866 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:36.808838    2556 generic.go:356] "Generic (PLEG): container finished" podID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerID="8421ff2132c98b6214f13dd0eef91d1820cbfc54b73c32d860d39074554c525c" exitCode=0
Jul 13 21:03:36.809211 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:36.808916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx" event={"ID":"0b969012-9179-4f98-bb13-0d5f146fdfaf","Type":"ContainerDied","Data":"8421ff2132c98b6214f13dd0eef91d1820cbfc54b73c32d860d39074554c525c"}
Jul 13 21:03:37.946646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:37.946627    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:38.078209 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.078125    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4fbc\" (UniqueName: \"kubernetes.io/projected/0b969012-9179-4f98-bb13-0d5f146fdfaf-kube-api-access-v4fbc\") pod \"0b969012-9179-4f98-bb13-0d5f146fdfaf\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") "
Jul 13 21:03:38.078384 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.078265    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-util\") pod \"0b969012-9179-4f98-bb13-0d5f146fdfaf\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") "
Jul 13 21:03:38.078384 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.078315    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-bundle\") pod \"0b969012-9179-4f98-bb13-0d5f146fdfaf\" (UID: \"0b969012-9179-4f98-bb13-0d5f146fdfaf\") "
Jul 13 21:03:38.078987 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.078963    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-bundle" (OuterVolumeSpecName: "bundle") pod "0b969012-9179-4f98-bb13-0d5f146fdfaf" (UID: "0b969012-9179-4f98-bb13-0d5f146fdfaf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:03:38.080290 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.080266    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b969012-9179-4f98-bb13-0d5f146fdfaf-kube-api-access-v4fbc" (OuterVolumeSpecName: "kube-api-access-v4fbc") pod "0b969012-9179-4f98-bb13-0d5f146fdfaf" (UID: "0b969012-9179-4f98-bb13-0d5f146fdfaf"). InnerVolumeSpecName "kube-api-access-v4fbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:03:38.083737 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.083704    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-util" (OuterVolumeSpecName: "util") pod "0b969012-9179-4f98-bb13-0d5f146fdfaf" (UID: "0b969012-9179-4f98-bb13-0d5f146fdfaf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:03:38.179924 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.179889    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-util\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:38.179924 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.179920    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b969012-9179-4f98-bb13-0d5f146fdfaf-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:38.179924 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.179929    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4fbc\" (UniqueName: \"kubernetes.io/projected/0b969012-9179-4f98-bb13-0d5f146fdfaf-kube-api-access-v4fbc\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:03:38.820518 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.820474    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx" event={"ID":"0b969012-9179-4f98-bb13-0d5f146fdfaf","Type":"ContainerDied","Data":"d04c69116af9305102c4fdccd3b7e7064a89687b826b51e14d6d54e5a909c938"}
Jul 13 21:03:38.820518 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.820522    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkm9vx"
Jul 13 21:03:38.820735 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:38.820532    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d04c69116af9305102c4fdccd3b7e7064a89687b826b51e14d6d54e5a909c938"
Jul 13 21:03:45.919123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919090    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"]
Jul 13 21:03:45.919529 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919440    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerName="extract"
Jul 13 21:03:45.919529 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919451    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerName="extract"
Jul 13 21:03:45.919529 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919468    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerName="util"
Jul 13 21:03:45.919529 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919474    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerName="util"
Jul 13 21:03:45.919529 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919486    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerName="pull"
Jul 13 21:03:45.919529 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919491    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerName="pull"
Jul 13 21:03:45.919712 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.919544    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b969012-9179-4f98-bb13-0d5f146fdfaf" containerName="extract"
Jul 13 21:03:45.922429 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.922414    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:45.926121 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.926065    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 21:03:45.926121 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.926088    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 21:03:45.926121 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.926064    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6zt5p\""
Jul 13 21:03:45.926355 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.926065    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 21:03:45.926398 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.926381    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 21:03:45.954379 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:45.954335    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"]
Jul 13 21:03:46.048075 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.048027    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f53550a6-a891-491c-b81e-2a39d2713b6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.048288 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.048115    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f53550a6-a891-491c-b81e-2a39d2713b6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.048288 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.048183    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f53550a6-a891-491c-b81e-2a39d2713b6c-manifests\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.048288 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.048249    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f53550a6-a891-491c-b81e-2a39d2713b6c-tmp\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.048503 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.048383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlc5t\" (UniqueName: \"kubernetes.io/projected/f53550a6-a891-491c-b81e-2a39d2713b6c-kube-api-access-jlc5t\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.149276 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.149250    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f53550a6-a891-491c-b81e-2a39d2713b6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.149276 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.149278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f53550a6-a891-491c-b81e-2a39d2713b6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.149519 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.149305    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f53550a6-a891-491c-b81e-2a39d2713b6c-manifests\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.149519 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.149369    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f53550a6-a891-491c-b81e-2a39d2713b6c-tmp\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.149519 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.149402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlc5t\" (UniqueName: \"kubernetes.io/projected/f53550a6-a891-491c-b81e-2a39d2713b6c-kube-api-access-jlc5t\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.149841 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.149818    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f53550a6-a891-491c-b81e-2a39d2713b6c-manifests\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.151626 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.151598    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f53550a6-a891-491c-b81e-2a39d2713b6c-tmp\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.151719 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.151678    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f53550a6-a891-491c-b81e-2a39d2713b6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.151719 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.151704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f53550a6-a891-491c-b81e-2a39d2713b6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.161845 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.161821    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlc5t\" (UniqueName: \"kubernetes.io/projected/f53550a6-a891-491c-b81e-2a39d2713b6c-kube-api-access-jlc5t\") pod \"opendatahub-operator-controller-manager-6fdc557c48-j4x96\" (UID: \"f53550a6-a891-491c-b81e-2a39d2713b6c\") " pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.233374 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.233308    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:46.379629 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.379607    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"]
Jul 13 21:03:46.381293 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:03:46.381267    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53550a6_a891_491c_b81e_2a39d2713b6c.slice/crio-80f9b6b85fe504e1891c228f4de370c8f4703180e4a250bc44a0919b7242f001 WatchSource:0}: Error finding container 80f9b6b85fe504e1891c228f4de370c8f4703180e4a250bc44a0919b7242f001: Status 404 returned error can't find the container with id 80f9b6b85fe504e1891c228f4de370c8f4703180e4a250bc44a0919b7242f001
Jul 13 21:03:46.859570 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:46.859540    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96" event={"ID":"f53550a6-a891-491c-b81e-2a39d2713b6c","Type":"ContainerStarted","Data":"80f9b6b85fe504e1891c228f4de370c8f4703180e4a250bc44a0919b7242f001"}
Jul 13 21:03:51.886786 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:51.886746    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96" event={"ID":"f53550a6-a891-491c-b81e-2a39d2713b6c","Type":"ContainerStarted","Data":"3ec565d3e04a5add145392b735e01f5c39d81c787b9c81d7e178499a55269a74"}
Jul 13 21:03:52.893221 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:52.893190    2556 generic.go:356] "Generic (PLEG): container finished" podID="f53550a6-a891-491c-b81e-2a39d2713b6c" containerID="3ec565d3e04a5add145392b735e01f5c39d81c787b9c81d7e178499a55269a74" exitCode=0
Jul 13 21:03:52.893663 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:52.893283    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96" event={"ID":"f53550a6-a891-491c-b81e-2a39d2713b6c","Type":"ContainerDied","Data":"3ec565d3e04a5add145392b735e01f5c39d81c787b9c81d7e178499a55269a74"}
Jul 13 21:03:53.899714 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:53.899685    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96" event={"ID":"f53550a6-a891-491c-b81e-2a39d2713b6c","Type":"ContainerStarted","Data":"bbc805f7e2ac6a74c780d66c7fe1589e75770ad9a3bd71d8dbc72934e9e84e3a"}
Jul 13 21:03:53.900107 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:53.899832    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:03:53.924474 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:53.924429    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96" podStartSLOduration=2.410120973 podStartE2EDuration="8.924415612s" totalImagesPullingTime="5.435325782s" totalInitContainerRuntime="1.078968857s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:03:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:03:53.922053853 +0000 UTC m=+594.860258164" watchObservedRunningTime="2026-07-13 21:03:53.924415612 +0000 UTC m=+594.862619923"
Jul 13 21:03:59.491418 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:59.491392    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:03:59.491821 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:03:59.491392    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:04:04.377639 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.377606    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"]
Jul 13 21:04:04.379818 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.379803    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.382535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.382509    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 13 21:04:04.382535 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.382521    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 13 21:04:04.382694 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.382523    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 13 21:04:04.382790 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.382773    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8xzqm\""
Jul 13 21:04:04.393154 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.393133    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"]
Jul 13 21:04:04.407763 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.407732    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67fef059-1701-44e2-9868-017589abe5b1-cert\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.407880 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.407767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb4lg\" (UniqueName: \"kubernetes.io/projected/67fef059-1701-44e2-9868-017589abe5b1-kube-api-access-cb4lg\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.407880 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.407806    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/67fef059-1701-44e2-9868-017589abe5b1-manager-config\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.407880 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.407851    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/67fef059-1701-44e2-9868-017589abe5b1-metrics-cert\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.509261 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.509227    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67fef059-1701-44e2-9868-017589abe5b1-cert\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.509261 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.509259    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb4lg\" (UniqueName: \"kubernetes.io/projected/67fef059-1701-44e2-9868-017589abe5b1-kube-api-access-cb4lg\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.509520 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.509281    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/67fef059-1701-44e2-9868-017589abe5b1-manager-config\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.509520 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.509393    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/67fef059-1701-44e2-9868-017589abe5b1-metrics-cert\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.509997 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.509977    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/67fef059-1701-44e2-9868-017589abe5b1-manager-config\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.511761 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.511742    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/67fef059-1701-44e2-9868-017589abe5b1-metrics-cert\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.511840 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.511793    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67fef059-1701-44e2-9868-017589abe5b1-cert\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.524153 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.524130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb4lg\" (UniqueName: \"kubernetes.io/projected/67fef059-1701-44e2-9868-017589abe5b1-kube-api-access-cb4lg\") pod \"lws-controller-manager-5f779bff78-r7hg4\" (UID: \"67fef059-1701-44e2-9868-017589abe5b1\") " pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.689245 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.689161    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:04.818537 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.818515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"]
Jul 13 21:04:04.820715 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:04:04.820679    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fef059_1701_44e2_9868_017589abe5b1.slice/crio-d01a8b50fda3141316bfdf4784098b95f42cd7778ea8140849c08a77bd160cc5 WatchSource:0}: Error finding container d01a8b50fda3141316bfdf4784098b95f42cd7778ea8140849c08a77bd160cc5: Status 404 returned error can't find the container with id d01a8b50fda3141316bfdf4784098b95f42cd7778ea8140849c08a77bd160cc5
Jul 13 21:04:04.907110 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.907078    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fdc557c48-j4x96"
Jul 13 21:04:04.959114 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:04.959008    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4" event={"ID":"67fef059-1701-44e2-9868-017589abe5b1","Type":"ContainerStarted","Data":"d01a8b50fda3141316bfdf4784098b95f42cd7778ea8140849c08a77bd160cc5"}
Jul 13 21:04:06.972810 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:06.972774    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4" event={"ID":"67fef059-1701-44e2-9868-017589abe5b1","Type":"ContainerStarted","Data":"da2d4e082a32a228fd98419f3c2e17c8a20a129759b6a2a0c92bf02084da3421"}
Jul 13 21:04:06.973299 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:06.972859    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:04:07.313563 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.313463    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4" podStartSLOduration=1.9143761769999998 podStartE2EDuration="3.313446306s" totalImagesPullingTime="1.399070129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:04:07.02721944 +0000 UTC m=+607.965423750" watchObservedRunningTime="2026-07-13 21:04:07.313446306 +0000 UTC m=+608.251650635"
Jul 13 21:04:07.313809 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.313793    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"]
Jul 13 21:04:07.316198 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.316174    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.319096 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.319077    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 21:04:07.320515 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.320501    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hshbr\""
Jul 13 21:04:07.321029 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.321013    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 21:04:07.331592 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.331565    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"]
Jul 13 21:04:07.337120 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.337100    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61856d71-02cf-46e3-8918-77a9df987dcb-tls-certs\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.337221 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.337180    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dhll\" (UniqueName: \"kubernetes.io/projected/61856d71-02cf-46e3-8918-77a9df987dcb-kube-api-access-7dhll\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.337221 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.337207    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61856d71-02cf-46e3-8918-77a9df987dcb-tmp\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.437748 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.437710    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61856d71-02cf-46e3-8918-77a9df987dcb-tls-certs\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.437943 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.437791    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dhll\" (UniqueName: \"kubernetes.io/projected/61856d71-02cf-46e3-8918-77a9df987dcb-kube-api-access-7dhll\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.437943 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.437812    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61856d71-02cf-46e3-8918-77a9df987dcb-tmp\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.440042 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.440013    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61856d71-02cf-46e3-8918-77a9df987dcb-tmp\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.440172 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.440154    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61856d71-02cf-46e3-8918-77a9df987dcb-tls-certs\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.451891 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.451867    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dhll\" (UniqueName: \"kubernetes.io/projected/61856d71-02cf-46e3-8918-77a9df987dcb-kube-api-access-7dhll\") pod \"kube-auth-proxy-5bf978c8f5-dbx8w\" (UID: \"61856d71-02cf-46e3-8918-77a9df987dcb\") " pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.626319 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.626290    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"
Jul 13 21:04:07.764396 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.764363    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w"]
Jul 13 21:04:07.766588 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:04:07.766563    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61856d71_02cf_46e3_8918_77a9df987dcb.slice/crio-6e4232455dd63ccc1ebf051afbe6add572b0733d4e1f589f398ab3ccc4180545 WatchSource:0}: Error finding container 6e4232455dd63ccc1ebf051afbe6add572b0733d4e1f589f398ab3ccc4180545: Status 404 returned error can't find the container with id 6e4232455dd63ccc1ebf051afbe6add572b0733d4e1f589f398ab3ccc4180545
Jul 13 21:04:07.978584 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:07.978494    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w" event={"ID":"61856d71-02cf-46e3-8918-77a9df987dcb","Type":"ContainerStarted","Data":"6e4232455dd63ccc1ebf051afbe6add572b0733d4e1f589f398ab3ccc4180545"}
Jul 13 21:04:10.994191 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:10.994112    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w" event={"ID":"61856d71-02cf-46e3-8918-77a9df987dcb","Type":"ContainerStarted","Data":"81ea4f8b9b7474c117b1e623370feaa398451413201c3990efaafc2d864ca8a1"}
Jul 13 21:04:11.012456 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:11.012404    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bf978c8f5-dbx8w" podStartSLOduration=1.073063131 podStartE2EDuration="4.012388929s" totalImagesPullingTime="2.939325798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:04:11.010117366 +0000 UTC m=+611.948321688" watchObservedRunningTime="2026-07-13 21:04:11.012388929 +0000 UTC m=+611.950593270"
Jul 13 21:04:17.981475 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:04:17.981443    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f779bff78-r7hg4"
Jul 13 21:05:23.368412 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.368381    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"]
Jul 13 21:05:23.371016 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.371001    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.373373 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.373354    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h6kj7\""
Jul 13 21:05:23.373468 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.373374    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 21:05:23.374313 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.374296    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 21:05:23.379799 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.379777    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"]
Jul 13 21:05:23.515843 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.515816    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.516005 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.515881    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.516005 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.515954    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqb78\" (UniqueName: \"kubernetes.io/projected/b543c256-f9a0-42c0-8dbc-f297fd076faf-kube-api-access-fqb78\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.617403 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.617370    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.617539 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.617414    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqb78\" (UniqueName: \"kubernetes.io/projected/b543c256-f9a0-42c0-8dbc-f297fd076faf-kube-api-access-fqb78\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.617539 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.617448    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.617713 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.617697    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.617753 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.617740    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.626002 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.625948    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqb78\" (UniqueName: \"kubernetes.io/projected/b543c256-f9a0-42c0-8dbc-f297fd076faf-kube-api-access-fqb78\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.682001 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.681982    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:23.807788 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.807767    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"]
Jul 13 21:05:23.809593 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:05:23.809566    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb543c256_f9a0_42c0_8dbc_f297fd076faf.slice/crio-a8d17e1dc92a35519d7429b1e4f8e3342e908bed8e8334c2f352f54b5250c565 WatchSource:0}: Error finding container a8d17e1dc92a35519d7429b1e4f8e3342e908bed8e8334c2f352f54b5250c565: Status 404 returned error can't find the container with id a8d17e1dc92a35519d7429b1e4f8e3342e908bed8e8334c2f352f54b5250c565
Jul 13 21:05:23.972731 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.972704    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"]
Jul 13 21:05:23.975120 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.975105    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:23.984521 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:23.984501    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"]
Jul 13 21:05:24.121639 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.121610    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.121792 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.121663    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.121792 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.121751    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25pfh\" (UniqueName: \"kubernetes.io/projected/7b775ec9-cc04-4ccf-9738-cb241f66886d-kube-api-access-25pfh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.222933 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.222861    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25pfh\" (UniqueName: \"kubernetes.io/projected/7b775ec9-cc04-4ccf-9738-cb241f66886d-kube-api-access-25pfh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.223054 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.222936    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.223054 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.222986    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.223364 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.223326    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.223405 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.223387    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.231306 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.231280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25pfh\" (UniqueName: \"kubernetes.io/projected/7b775ec9-cc04-4ccf-9738-cb241f66886d-kube-api-access-25pfh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.284184 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.284161    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:24.358199 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.358169    2556 generic.go:356] "Generic (PLEG): container finished" podID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerID="8eeb5e06f2944abee8e5e81bf66802ffa8b80a1a64380905a7224d8fac7e3cf8" exitCode=0
Jul 13 21:05:24.358318 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.358236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm" event={"ID":"b543c256-f9a0-42c0-8dbc-f297fd076faf","Type":"ContainerDied","Data":"8eeb5e06f2944abee8e5e81bf66802ffa8b80a1a64380905a7224d8fac7e3cf8"}
Jul 13 21:05:24.358318 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.358283    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm" event={"ID":"b543c256-f9a0-42c0-8dbc-f297fd076faf","Type":"ContainerStarted","Data":"a8d17e1dc92a35519d7429b1e4f8e3342e908bed8e8334c2f352f54b5250c565"}
Jul 13 21:05:24.358777 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.358763    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:05:24.413468 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.413447    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"]
Jul 13 21:05:24.415233 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:05:24.415203    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b775ec9_cc04_4ccf_9738_cb241f66886d.slice/crio-d9de8cec7b6de9ae401eb52ed2d7134c8e73ce6d8beb509f4185f20929a2cea2 WatchSource:0}: Error finding container d9de8cec7b6de9ae401eb52ed2d7134c8e73ce6d8beb509f4185f20929a2cea2: Status 404 returned error can't find the container with id d9de8cec7b6de9ae401eb52ed2d7134c8e73ce6d8beb509f4185f20929a2cea2
Jul 13 21:05:24.581222 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.578647    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"]
Jul 13 21:05:24.583082 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.583060    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.588444 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.588424    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"]
Jul 13 21:05:24.727610 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.727570    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjz4t\" (UniqueName: \"kubernetes.io/projected/f36704e8-6638-4cd9-8fa8-a6a10512ba21-kube-api-access-bjz4t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.727742 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.727627    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.727742 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.727691    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.828398 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.828285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjz4t\" (UniqueName: \"kubernetes.io/projected/f36704e8-6638-4cd9-8fa8-a6a10512ba21-kube-api-access-bjz4t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.828398 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.828367    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.828596 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.828410    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.828783 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.828764    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.828849 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.828799    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.836804 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.836787    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjz4t\" (UniqueName: \"kubernetes.io/projected/f36704e8-6638-4cd9-8fa8-a6a10512ba21-kube-api-access-bjz4t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.893281 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.893253    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:24.971975 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.971944    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"]
Jul 13 21:05:24.975491 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.975473    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:24.985103 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:24.985038    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"]
Jul 13 21:05:25.028620 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.028597    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"]
Jul 13 21:05:25.029805 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:05:25.029780    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36704e8_6638_4cd9_8fa8_a6a10512ba21.slice/crio-df3f8fe264c4870a4994531ed368ffb8e54b992d15c4f7e1945f02b6e55c4356 WatchSource:0}: Error finding container df3f8fe264c4870a4994531ed368ffb8e54b992d15c4f7e1945f02b6e55c4356: Status 404 returned error can't find the container with id df3f8fe264c4870a4994531ed368ffb8e54b992d15c4f7e1945f02b6e55c4356
Jul 13 21:05:25.130773 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.130747    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.130871 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.130811    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxgz5\" (UniqueName: \"kubernetes.io/projected/e505efe2-5353-4072-a357-38c172513a8f-kube-api-access-wxgz5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.130871 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.130860    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.231972 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.231944    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.232143 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.232040    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.232143 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.232089    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxgz5\" (UniqueName: \"kubernetes.io/projected/e505efe2-5353-4072-a357-38c172513a8f-kube-api-access-wxgz5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.232319 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.232299    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.232425 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.232394    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.240716 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.240697    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxgz5\" (UniqueName: \"kubernetes.io/projected/e505efe2-5353-4072-a357-38c172513a8f-kube-api-access-wxgz5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.289461 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.289439    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:25.365448 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.365414    2556 generic.go:356] "Generic (PLEG): container finished" podID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerID="51b013e35c3a039e3fdd0f2b0837bd512efb54f536516628dee3f0c8e12c1610" exitCode=0
Jul 13 21:05:25.365591 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.365494    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp" event={"ID":"7b775ec9-cc04-4ccf-9738-cb241f66886d","Type":"ContainerDied","Data":"51b013e35c3a039e3fdd0f2b0837bd512efb54f536516628dee3f0c8e12c1610"}
Jul 13 21:05:25.365591 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.365541    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp" event={"ID":"7b775ec9-cc04-4ccf-9738-cb241f66886d","Type":"ContainerStarted","Data":"d9de8cec7b6de9ae401eb52ed2d7134c8e73ce6d8beb509f4185f20929a2cea2"}
Jul 13 21:05:25.367289 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.367263    2556 generic.go:356] "Generic (PLEG): container finished" podID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerID="1ff457f7c1cf6e7ceaab550cccf464333a107e5c32bc3556bd2d6f2a0a0fb1d6" exitCode=0
Jul 13 21:05:25.367436 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.367304    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75" event={"ID":"f36704e8-6638-4cd9-8fa8-a6a10512ba21","Type":"ContainerDied","Data":"1ff457f7c1cf6e7ceaab550cccf464333a107e5c32bc3556bd2d6f2a0a0fb1d6"}
Jul 13 21:05:25.367436 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.367359    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75" event={"ID":"f36704e8-6638-4cd9-8fa8-a6a10512ba21","Type":"ContainerStarted","Data":"df3f8fe264c4870a4994531ed368ffb8e54b992d15c4f7e1945f02b6e55c4356"}
Jul 13 21:05:25.421654 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:25.421616    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"]
Jul 13 21:05:25.449614 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:05:25.449582    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode505efe2_5353_4072_a357_38c172513a8f.slice/crio-46efa6eb305c9f8117fadf4df36362805493a0010becb528ddcf735078befa08 WatchSource:0}: Error finding container 46efa6eb305c9f8117fadf4df36362805493a0010becb528ddcf735078befa08: Status 404 returned error can't find the container with id 46efa6eb305c9f8117fadf4df36362805493a0010becb528ddcf735078befa08
Jul 13 21:05:26.374729 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.374702    2556 generic.go:356] "Generic (PLEG): container finished" podID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerID="56f6dc0376701d11885c2096a38fdc6d9651cc3339edb61117d8e3f75fc841e4" exitCode=0
Jul 13 21:05:26.374843 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.374782    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75" event={"ID":"f36704e8-6638-4cd9-8fa8-a6a10512ba21","Type":"ContainerDied","Data":"56f6dc0376701d11885c2096a38fdc6d9651cc3339edb61117d8e3f75fc841e4"}
Jul 13 21:05:26.376635 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.376614    2556 generic.go:356] "Generic (PLEG): container finished" podID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerID="8f7b397af5b70ded5b18ed25b4ecc16458339a5ac8e3e83524f471fa5cfdcfa2" exitCode=0
Jul 13 21:05:26.376742 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.376701    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp" event={"ID":"7b775ec9-cc04-4ccf-9738-cb241f66886d","Type":"ContainerDied","Data":"8f7b397af5b70ded5b18ed25b4ecc16458339a5ac8e3e83524f471fa5cfdcfa2"}
Jul 13 21:05:26.378603 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.378579    2556 generic.go:356] "Generic (PLEG): container finished" podID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerID="9fc8e5ef1dd4f6543c7e96ad5a087fec5793a09397d6711ef17c88afe38232e5" exitCode=0
Jul 13 21:05:26.378698 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.378686    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm" event={"ID":"b543c256-f9a0-42c0-8dbc-f297fd076faf","Type":"ContainerDied","Data":"9fc8e5ef1dd4f6543c7e96ad5a087fec5793a09397d6711ef17c88afe38232e5"}
Jul 13 21:05:26.380092 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.380070    2556 generic.go:356] "Generic (PLEG): container finished" podID="e505efe2-5353-4072-a357-38c172513a8f" containerID="68bfa78e3e5ead3484e343f8addc46241564c9ccef049a6350e030abcc988dbb" exitCode=0
Jul 13 21:05:26.380178 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.380115    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf" event={"ID":"e505efe2-5353-4072-a357-38c172513a8f","Type":"ContainerDied","Data":"68bfa78e3e5ead3484e343f8addc46241564c9ccef049a6350e030abcc988dbb"}
Jul 13 21:05:26.380178 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:26.380147    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf" event={"ID":"e505efe2-5353-4072-a357-38c172513a8f","Type":"ContainerStarted","Data":"46efa6eb305c9f8117fadf4df36362805493a0010becb528ddcf735078befa08"}
Jul 13 21:05:27.386613 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:27.386590    2556 generic.go:356] "Generic (PLEG): container finished" podID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerID="c09c271795b3820ca9da4c288f315ab8645966872a20f4dbd669331273311345" exitCode=0
Jul 13 21:05:27.386956 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:27.386670    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75" event={"ID":"f36704e8-6638-4cd9-8fa8-a6a10512ba21","Type":"ContainerDied","Data":"c09c271795b3820ca9da4c288f315ab8645966872a20f4dbd669331273311345"}
Jul 13 21:05:27.388707 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:27.388687    2556 generic.go:356] "Generic (PLEG): container finished" podID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerID="9c4090816b4ace9f3002085bb9d0690c7a16392959c087b532da8ac65a68e29a" exitCode=0
Jul 13 21:05:27.388805 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:27.388729    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp" event={"ID":"7b775ec9-cc04-4ccf-9738-cb241f66886d","Type":"ContainerDied","Data":"9c4090816b4ace9f3002085bb9d0690c7a16392959c087b532da8ac65a68e29a"}
Jul 13 21:05:27.390683 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:27.390663    2556 generic.go:356] "Generic (PLEG): container finished" podID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerID="8504219d5de6f08e2b0021b7fd0f27f6c6f9011f7ee4c8d18f453cb97e762b2a" exitCode=0
Jul 13 21:05:27.390770 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:27.390732    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm" event={"ID":"b543c256-f9a0-42c0-8dbc-f297fd076faf","Type":"ContainerDied","Data":"8504219d5de6f08e2b0021b7fd0f27f6c6f9011f7ee4c8d18f453cb97e762b2a"}
Jul 13 21:05:28.398212 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.398177    2556 generic.go:356] "Generic (PLEG): container finished" podID="e505efe2-5353-4072-a357-38c172513a8f" containerID="e370ed573ae64bc52d2624bb4dcf10847a74e0cf950b9d1e05aaebd8c324e81d" exitCode=0
Jul 13 21:05:28.398644 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.398258    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf" event={"ID":"e505efe2-5353-4072-a357-38c172513a8f","Type":"ContainerDied","Data":"e370ed573ae64bc52d2624bb4dcf10847a74e0cf950b9d1e05aaebd8c324e81d"}
Jul 13 21:05:28.563115 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.563088    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:28.646421 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.646402    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:28.649378 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.649324    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:28.662006 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.661990    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-util\") pod \"7b775ec9-cc04-4ccf-9738-cb241f66886d\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") "
Jul 13 21:05:28.662113 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.662071    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-bundle\") pod \"7b775ec9-cc04-4ccf-9738-cb241f66886d\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") "
Jul 13 21:05:28.662170 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.662156    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25pfh\" (UniqueName: \"kubernetes.io/projected/7b775ec9-cc04-4ccf-9738-cb241f66886d-kube-api-access-25pfh\") pod \"7b775ec9-cc04-4ccf-9738-cb241f66886d\" (UID: \"7b775ec9-cc04-4ccf-9738-cb241f66886d\") "
Jul 13 21:05:28.662682 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.662651    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-bundle" (OuterVolumeSpecName: "bundle") pod "7b775ec9-cc04-4ccf-9738-cb241f66886d" (UID: "7b775ec9-cc04-4ccf-9738-cb241f66886d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:28.664239 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.664217    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b775ec9-cc04-4ccf-9738-cb241f66886d-kube-api-access-25pfh" (OuterVolumeSpecName: "kube-api-access-25pfh") pod "7b775ec9-cc04-4ccf-9738-cb241f66886d" (UID: "7b775ec9-cc04-4ccf-9738-cb241f66886d"). InnerVolumeSpecName "kube-api-access-25pfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:05:28.667384 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.667359    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-util" (OuterVolumeSpecName: "util") pod "7b775ec9-cc04-4ccf-9738-cb241f66886d" (UID: "7b775ec9-cc04-4ccf-9738-cb241f66886d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:28.763114 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763088    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-bundle\") pod \"b543c256-f9a0-42c0-8dbc-f297fd076faf\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") "
Jul 13 21:05:28.763300 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763151    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqb78\" (UniqueName: \"kubernetes.io/projected/b543c256-f9a0-42c0-8dbc-f297fd076faf-kube-api-access-fqb78\") pod \"b543c256-f9a0-42c0-8dbc-f297fd076faf\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") "
Jul 13 21:05:28.763300 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763183    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-bundle\") pod \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") "
Jul 13 21:05:28.763300 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763209    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-util\") pod \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") "
Jul 13 21:05:28.763524 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763389    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjz4t\" (UniqueName: \"kubernetes.io/projected/f36704e8-6638-4cd9-8fa8-a6a10512ba21-kube-api-access-bjz4t\") pod \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\" (UID: \"f36704e8-6638-4cd9-8fa8-a6a10512ba21\") "
Jul 13 21:05:28.763524 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763467    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-util\") pod \"b543c256-f9a0-42c0-8dbc-f297fd076faf\" (UID: \"b543c256-f9a0-42c0-8dbc-f297fd076faf\") "
Jul 13 21:05:28.763623 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763599    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-bundle" (OuterVolumeSpecName: "bundle") pod "b543c256-f9a0-42c0-8dbc-f297fd076faf" (UID: "b543c256-f9a0-42c0-8dbc-f297fd076faf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:28.763839 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763769    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25pfh\" (UniqueName: \"kubernetes.io/projected/7b775ec9-cc04-4ccf-9738-cb241f66886d-kube-api-access-25pfh\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.763839 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763792    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-util\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.763839 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763806    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.763839 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763818    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b775ec9-cc04-4ccf-9738-cb241f66886d-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.764092 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.763827    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-bundle" (OuterVolumeSpecName: "bundle") pod "f36704e8-6638-4cd9-8fa8-a6a10512ba21" (UID: "f36704e8-6638-4cd9-8fa8-a6a10512ba21"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:28.765409 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.765382    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b543c256-f9a0-42c0-8dbc-f297fd076faf-kube-api-access-fqb78" (OuterVolumeSpecName: "kube-api-access-fqb78") pod "b543c256-f9a0-42c0-8dbc-f297fd076faf" (UID: "b543c256-f9a0-42c0-8dbc-f297fd076faf"). InnerVolumeSpecName "kube-api-access-fqb78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:05:28.765570 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.765552    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36704e8-6638-4cd9-8fa8-a6a10512ba21-kube-api-access-bjz4t" (OuterVolumeSpecName: "kube-api-access-bjz4t") pod "f36704e8-6638-4cd9-8fa8-a6a10512ba21" (UID: "f36704e8-6638-4cd9-8fa8-a6a10512ba21"). InnerVolumeSpecName "kube-api-access-bjz4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:05:28.769372 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.769352    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-util" (OuterVolumeSpecName: "util") pod "b543c256-f9a0-42c0-8dbc-f297fd076faf" (UID: "b543c256-f9a0-42c0-8dbc-f297fd076faf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:28.769517 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.769497    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-util" (OuterVolumeSpecName: "util") pod "f36704e8-6638-4cd9-8fa8-a6a10512ba21" (UID: "f36704e8-6638-4cd9-8fa8-a6a10512ba21"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:28.864431 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.864408    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-util\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.864431 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.864428    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjz4t\" (UniqueName: \"kubernetes.io/projected/f36704e8-6638-4cd9-8fa8-a6a10512ba21-kube-api-access-bjz4t\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.864561 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.864438    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b543c256-f9a0-42c0-8dbc-f297fd076faf-util\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.864561 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.864452    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqb78\" (UniqueName: \"kubernetes.io/projected/b543c256-f9a0-42c0-8dbc-f297fd076faf-kube-api-access-fqb78\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:28.864561 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:28.864464    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f36704e8-6638-4cd9-8fa8-a6a10512ba21-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:29.405281 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.405255    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75"
Jul 13 21:05:29.405281 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.405258    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75" event={"ID":"f36704e8-6638-4cd9-8fa8-a6a10512ba21","Type":"ContainerDied","Data":"df3f8fe264c4870a4994531ed368ffb8e54b992d15c4f7e1945f02b6e55c4356"}
Jul 13 21:05:29.405824 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.405303    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df3f8fe264c4870a4994531ed368ffb8e54b992d15c4f7e1945f02b6e55c4356"
Jul 13 21:05:29.407250 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.407227    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp"
Jul 13 21:05:29.407250 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.407219    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp" event={"ID":"7b775ec9-cc04-4ccf-9738-cb241f66886d","Type":"ContainerDied","Data":"d9de8cec7b6de9ae401eb52ed2d7134c8e73ce6d8beb509f4185f20929a2cea2"}
Jul 13 21:05:29.407455 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.407273    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9de8cec7b6de9ae401eb52ed2d7134c8e73ce6d8beb509f4185f20929a2cea2"
Jul 13 21:05:29.409242 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.409226    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm"
Jul 13 21:05:29.409385 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.409247    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm" event={"ID":"b543c256-f9a0-42c0-8dbc-f297fd076faf","Type":"ContainerDied","Data":"a8d17e1dc92a35519d7429b1e4f8e3342e908bed8e8334c2f352f54b5250c565"}
Jul 13 21:05:29.409385 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.409283    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8d17e1dc92a35519d7429b1e4f8e3342e908bed8e8334c2f352f54b5250c565"
Jul 13 21:05:29.411471 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.411445    2556 generic.go:356] "Generic (PLEG): container finished" podID="e505efe2-5353-4072-a357-38c172513a8f" containerID="e11fafff030b4aada8adc64ecf9e6d350d498ed491ddeddce2cc709cbdd1f3ab" exitCode=0
Jul 13 21:05:29.411632 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:29.411516    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf" event={"ID":"e505efe2-5353-4072-a357-38c172513a8f","Type":"ContainerDied","Data":"e11fafff030b4aada8adc64ecf9e6d350d498ed491ddeddce2cc709cbdd1f3ab"}
Jul 13 21:05:30.556980 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.556955    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:30.578038 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.578008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-bundle\") pod \"e505efe2-5353-4072-a357-38c172513a8f\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") "
Jul 13 21:05:30.578192 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.578073    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-util\") pod \"e505efe2-5353-4072-a357-38c172513a8f\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") "
Jul 13 21:05:30.578192 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.578164    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxgz5\" (UniqueName: \"kubernetes.io/projected/e505efe2-5353-4072-a357-38c172513a8f-kube-api-access-wxgz5\") pod \"e505efe2-5353-4072-a357-38c172513a8f\" (UID: \"e505efe2-5353-4072-a357-38c172513a8f\") "
Jul 13 21:05:30.578605 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.578579    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-bundle" (OuterVolumeSpecName: "bundle") pod "e505efe2-5353-4072-a357-38c172513a8f" (UID: "e505efe2-5353-4072-a357-38c172513a8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:30.580254 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.580230    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e505efe2-5353-4072-a357-38c172513a8f-kube-api-access-wxgz5" (OuterVolumeSpecName: "kube-api-access-wxgz5") pod "e505efe2-5353-4072-a357-38c172513a8f" (UID: "e505efe2-5353-4072-a357-38c172513a8f"). InnerVolumeSpecName "kube-api-access-wxgz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:05:30.583687 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.583669    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-util" (OuterVolumeSpecName: "util") pod "e505efe2-5353-4072-a357-38c172513a8f" (UID: "e505efe2-5353-4072-a357-38c172513a8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:05:30.678885 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.678806    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxgz5\" (UniqueName: \"kubernetes.io/projected/e505efe2-5353-4072-a357-38c172513a8f-kube-api-access-wxgz5\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:30.678885 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.678835    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:30.678885 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:30.678843    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e505efe2-5353-4072-a357-38c172513a8f-util\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:31.426215 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:31.426181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf" event={"ID":"e505efe2-5353-4072-a357-38c172513a8f","Type":"ContainerDied","Data":"46efa6eb305c9f8117fadf4df36362805493a0010becb528ddcf735078befa08"}
Jul 13 21:05:31.426215 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:31.426209    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf"
Jul 13 21:05:31.426433 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:31.426251    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46efa6eb305c9f8117fadf4df36362805493a0010becb528ddcf735078befa08"
Jul 13 21:05:31.591209 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:31.591178    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db5d69799-85nfw"]
Jul 13 21:05:56.622057 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:56.621951    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5db5d69799-85nfw" podUID="9623e620-2c82-4355-868c-bdf48691a417" containerName="console" containerID="cri-o://de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100" gracePeriod=15
Jul 13 21:05:56.870499 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:56.870477    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:05:57.001688 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.001611    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-oauth-serving-cert\") pod \"9623e620-2c82-4355-868c-bdf48691a417\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") "
Jul 13 21:05:57.001688 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.001642    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-serving-cert\") pod \"9623e620-2c82-4355-868c-bdf48691a417\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") "
Jul 13 21:05:57.001688 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.001658    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-trusted-ca-bundle\") pod \"9623e620-2c82-4355-868c-bdf48691a417\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") "
Jul 13 21:05:57.001688 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.001677    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-service-ca\") pod \"9623e620-2c82-4355-868c-bdf48691a417\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") "
Jul 13 21:05:57.002053 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.001717    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv72m\" (UniqueName: \"kubernetes.io/projected/9623e620-2c82-4355-868c-bdf48691a417-kube-api-access-nv72m\") pod \"9623e620-2c82-4355-868c-bdf48691a417\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") "
Jul 13 21:05:57.002053 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.001742    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-oauth-config\") pod \"9623e620-2c82-4355-868c-bdf48691a417\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") "
Jul 13 21:05:57.002053 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.001788    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-console-config\") pod \"9623e620-2c82-4355-868c-bdf48691a417\" (UID: \"9623e620-2c82-4355-868c-bdf48691a417\") "
Jul 13 21:05:57.002203 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.002050    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9623e620-2c82-4355-868c-bdf48691a417" (UID: "9623e620-2c82-4355-868c-bdf48691a417"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:05:57.002203 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.002079    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9623e620-2c82-4355-868c-bdf48691a417" (UID: "9623e620-2c82-4355-868c-bdf48691a417"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:05:57.002296 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.002268    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-console-config" (OuterVolumeSpecName: "console-config") pod "9623e620-2c82-4355-868c-bdf48691a417" (UID: "9623e620-2c82-4355-868c-bdf48691a417"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:05:57.002384 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.002364    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-service-ca" (OuterVolumeSpecName: "service-ca") pod "9623e620-2c82-4355-868c-bdf48691a417" (UID: "9623e620-2c82-4355-868c-bdf48691a417"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:05:57.003959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.003931    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9623e620-2c82-4355-868c-bdf48691a417" (UID: "9623e620-2c82-4355-868c-bdf48691a417"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:05:57.004366 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.004313    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9623e620-2c82-4355-868c-bdf48691a417" (UID: "9623e620-2c82-4355-868c-bdf48691a417"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:05:57.004366 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.004327    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9623e620-2c82-4355-868c-bdf48691a417-kube-api-access-nv72m" (OuterVolumeSpecName: "kube-api-access-nv72m") pod "9623e620-2c82-4355-868c-bdf48691a417" (UID: "9623e620-2c82-4355-868c-bdf48691a417"). InnerVolumeSpecName "kube-api-access-nv72m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:05:57.103059 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.103032    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-oauth-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:57.103059 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.103055    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-serving-cert\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:57.103059 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.103064    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-trusted-ca-bundle\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:57.103252 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.103072    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-service-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:57.103252 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.103080    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv72m\" (UniqueName: \"kubernetes.io/projected/9623e620-2c82-4355-868c-bdf48691a417-kube-api-access-nv72m\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:57.103252 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.103088    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9623e620-2c82-4355-868c-bdf48691a417-console-oauth-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:57.103252 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.103096    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9623e620-2c82-4355-868c-bdf48691a417-console-config\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:05:57.564150 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.564115    2556 generic.go:356] "Generic (PLEG): container finished" podID="9623e620-2c82-4355-868c-bdf48691a417" containerID="de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100" exitCode=0
Jul 13 21:05:57.564326 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.564164    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db5d69799-85nfw"
Jul 13 21:05:57.564326 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.564162    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db5d69799-85nfw" event={"ID":"9623e620-2c82-4355-868c-bdf48691a417","Type":"ContainerDied","Data":"de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100"}
Jul 13 21:05:57.564326 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.564287    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db5d69799-85nfw" event={"ID":"9623e620-2c82-4355-868c-bdf48691a417","Type":"ContainerDied","Data":"5c536c31149e7892f48cf92ecb33ee8fb8fbcdb38eb1c9a2913a6180ef4d381b"}
Jul 13 21:05:57.564326 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.564315    2556 scope.go:117] "RemoveContainer" containerID="de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100"
Jul 13 21:05:57.573800 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.573783    2556 scope.go:117] "RemoveContainer" containerID="de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100"
Jul 13 21:05:57.574037 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:05:57.574018    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100\": container with ID starting with de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100 not found: ID does not exist" containerID="de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100"
Jul 13 21:05:57.574081 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.574045    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100"} err="failed to get container status \"de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100\": rpc error: code = NotFound desc = could not find container \"de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100\": container with ID starting with de66412274d9a4aede53221ea68aea2f691c4d722f8df8fd282e3a088015d100 not found: ID does not exist"
Jul 13 21:05:57.585451 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.585427    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db5d69799-85nfw"]
Jul 13 21:05:57.589815 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.589798    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5db5d69799-85nfw"]
Jul 13 21:05:57.602172 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:05:57.602148    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9623e620-2c82-4355-868c-bdf48691a417" path="/var/lib/kubelet/pods/9623e620-2c82-4355-868c-bdf48691a417/volumes"
Jul 13 21:06:00.859006 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.858974    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"]
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859316    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9623e620-2c82-4355-868c-bdf48691a417" containerName="console"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859326    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9623e620-2c82-4355-868c-bdf48691a417" containerName="console"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859353    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e505efe2-5353-4072-a357-38c172513a8f" containerName="util"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859358    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e505efe2-5353-4072-a357-38c172513a8f" containerName="util"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859365    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e505efe2-5353-4072-a357-38c172513a8f" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859370    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e505efe2-5353-4072-a357-38c172513a8f" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859380    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859385    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859393    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859398    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859403    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerName="util"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859410    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerName="util"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859421    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerName="util"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859426    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerName="util"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859433    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859437    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerName="pull"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859443    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerName="extract"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859447    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerName="extract"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859454    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerName="extract"
Jul 13 21:06:00.859449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859458    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859465    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e505efe2-5353-4072-a357-38c172513a8f" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859470    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e505efe2-5353-4072-a357-38c172513a8f" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859477    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerName="util"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859481    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerName="util"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859489    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859493    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859559    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b775ec9-cc04-4ccf-9738-cb241f66886d" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859566    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f36704e8-6638-4cd9-8fa8-a6a10512ba21" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859573    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9623e620-2c82-4355-868c-bdf48691a417" containerName="console"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859579    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b543c256-f9a0-42c0-8dbc-f297fd076faf" containerName="extract"
Jul 13 21:06:00.860085 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.859585    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e505efe2-5353-4072-a357-38c172513a8f" containerName="extract"
Jul 13 21:06:00.864015 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.863999    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:00.866587 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.866564    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 21:06:00.866740 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.866640    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 13 21:06:00.866740 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.866703    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 13 21:06:00.867558 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.867543    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h6kj7\""
Jul 13 21:06:00.867633 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.867575    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 21:06:00.874442 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.874425    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"]
Jul 13 21:06:00.937153 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.937123    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtrj\" (UniqueName: \"kubernetes.io/projected/e51b2976-ac19-413c-aab1-c35e6096060c-kube-api-access-crtrj\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:00.937270 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.937162    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e51b2976-ac19-413c-aab1-c35e6096060c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:00.937270 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:00.937253    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e51b2976-ac19-413c-aab1-c35e6096060c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.038523 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.038497    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtrj\" (UniqueName: \"kubernetes.io/projected/e51b2976-ac19-413c-aab1-c35e6096060c-kube-api-access-crtrj\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.038623 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.038533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e51b2976-ac19-413c-aab1-c35e6096060c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.038623 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.038575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e51b2976-ac19-413c-aab1-c35e6096060c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.038716 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:06:01.038700    2556 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 13 21:06:01.038783 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:06:01.038772    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e51b2976-ac19-413c-aab1-c35e6096060c-plugin-serving-cert podName:e51b2976-ac19-413c-aab1-c35e6096060c nodeName:}" failed. No retries permitted until 2026-07-13 21:06:01.538754306 +0000 UTC m=+722.476958597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e51b2976-ac19-413c-aab1-c35e6096060c-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-7cgn9" (UID: "e51b2976-ac19-413c-aab1-c35e6096060c") : secret "plugin-serving-cert" not found
Jul 13 21:06:01.039123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.039107    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e51b2976-ac19-413c-aab1-c35e6096060c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.077562 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.077543    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtrj\" (UniqueName: \"kubernetes.io/projected/e51b2976-ac19-413c-aab1-c35e6096060c-kube-api-access-crtrj\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.542326 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.542282    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e51b2976-ac19-413c-aab1-c35e6096060c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.544655 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.544627    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e51b2976-ac19-413c-aab1-c35e6096060c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7cgn9\" (UID: \"e51b2976-ac19-413c-aab1-c35e6096060c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.774423 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.774379    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"
Jul 13 21:06:01.904638 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:01.904613    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9"]
Jul 13 21:06:01.906554 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:06:01.906529    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode51b2976_ac19_413c_aab1_c35e6096060c.slice/crio-af650f90e0c9b5fef4d6479d2c26b81260a907d3a31a3df75f80ccd7d86fb64f WatchSource:0}: Error finding container af650f90e0c9b5fef4d6479d2c26b81260a907d3a31a3df75f80ccd7d86fb64f: Status 404 returned error can't find the container with id af650f90e0c9b5fef4d6479d2c26b81260a907d3a31a3df75f80ccd7d86fb64f
Jul 13 21:06:02.590895 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:02.590859    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9" event={"ID":"e51b2976-ac19-413c-aab1-c35e6096060c","Type":"ContainerStarted","Data":"af650f90e0c9b5fef4d6479d2c26b81260a907d3a31a3df75f80ccd7d86fb64f"}
Jul 13 21:06:26.742124 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:26.742092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9" event={"ID":"e51b2976-ac19-413c-aab1-c35e6096060c","Type":"ContainerStarted","Data":"800c12eedd085dc0fa346149e48d4f06f610d1336a4412cd6c766a52890dae79"}
Jul 13 21:06:26.765478 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:26.765431    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7cgn9" podStartSLOduration=2.653180049 podStartE2EDuration="26.765415177s" totalImagesPullingTime="24.112235128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:06:26.764589886 +0000 UTC m=+747.702794215" watchObservedRunningTime="2026-07-13 21:06:26.765415177 +0000 UTC m=+747.703619488"
Jul 13 21:06:52.367935 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.367906    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:06:52.462562 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.462525    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:06:52.462562 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.462562    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:06:52.462767 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.462647    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.465038 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.465014    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 13 21:06:52.598248 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.598222    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a-config-file\") pod \"limitador-limitador-54d6d988d5-sb5qt\" (UID: \"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.598406 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.598255    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t82jz\" (UniqueName: \"kubernetes.io/projected/d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a-kube-api-access-t82jz\") pod \"limitador-limitador-54d6d988d5-sb5qt\" (UID: \"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.699171 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.699098    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a-config-file\") pod \"limitador-limitador-54d6d988d5-sb5qt\" (UID: \"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.699171 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.699140    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t82jz\" (UniqueName: \"kubernetes.io/projected/d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a-kube-api-access-t82jz\") pod \"limitador-limitador-54d6d988d5-sb5qt\" (UID: \"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.699737 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.699718    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a-config-file\") pod \"limitador-limitador-54d6d988d5-sb5qt\" (UID: \"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.707203 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.707179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t82jz\" (UniqueName: \"kubernetes.io/projected/d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a-kube-api-access-t82jz\") pod \"limitador-limitador-54d6d988d5-sb5qt\" (UID: \"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.770513 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.770482    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-z5w5k"]
Jul 13 21:06:52.772063 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.772044    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:52.941121 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.941084    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-z5w5k"]
Jul 13 21:06:52.941284 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.941188    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z5w5k"
Jul 13 21:06:52.943659 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:52.943632    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7msf9\""
Jul 13 21:06:53.073758 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.073730    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:06:53.074644 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:06:53.074619    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9197f55_c9c8_4c47_a9ca_c9e2d2a4132a.slice/crio-cf082b8f5c4bf3610756d0463089030fbdf6953405e48d65fc03f5cb4dab411e WatchSource:0}: Error finding container cf082b8f5c4bf3610756d0463089030fbdf6953405e48d65fc03f5cb4dab411e: Status 404 returned error can't find the container with id cf082b8f5c4bf3610756d0463089030fbdf6953405e48d65fc03f5cb4dab411e
Jul 13 21:06:53.103732 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.103703    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w95g5\" (UniqueName: \"kubernetes.io/projected/3dceb5de-c7b5-4e22-b363-95123272ff81-kube-api-access-w95g5\") pod \"authorino-58cd955675-z5w5k\" (UID: \"3dceb5de-c7b5-4e22-b363-95123272ff81\") " pod="kuadrant-system/authorino-58cd955675-z5w5k"
Jul 13 21:06:53.204765 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.204706    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w95g5\" (UniqueName: \"kubernetes.io/projected/3dceb5de-c7b5-4e22-b363-95123272ff81-kube-api-access-w95g5\") pod \"authorino-58cd955675-z5w5k\" (UID: \"3dceb5de-c7b5-4e22-b363-95123272ff81\") " pod="kuadrant-system/authorino-58cd955675-z5w5k"
Jul 13 21:06:53.212897 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.212878    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w95g5\" (UniqueName: \"kubernetes.io/projected/3dceb5de-c7b5-4e22-b363-95123272ff81-kube-api-access-w95g5\") pod \"authorino-58cd955675-z5w5k\" (UID: \"3dceb5de-c7b5-4e22-b363-95123272ff81\") " pod="kuadrant-system/authorino-58cd955675-z5w5k"
Jul 13 21:06:53.266856 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.266836    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z5w5k"
Jul 13 21:06:53.430647 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.430622    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-z5w5k"]
Jul 13 21:06:53.432200 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:06:53.432175    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dceb5de_c7b5_4e22_b363_95123272ff81.slice/crio-6b9cb1cd36e02ad7c1d6229a49f8fde86ca38ec88f205a7cdf98ff7fdd962cd6 WatchSource:0}: Error finding container 6b9cb1cd36e02ad7c1d6229a49f8fde86ca38ec88f205a7cdf98ff7fdd962cd6: Status 404 returned error can't find the container with id 6b9cb1cd36e02ad7c1d6229a49f8fde86ca38ec88f205a7cdf98ff7fdd962cd6
Jul 13 21:06:53.887567 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.887528    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z5w5k" event={"ID":"3dceb5de-c7b5-4e22-b363-95123272ff81","Type":"ContainerStarted","Data":"6b9cb1cd36e02ad7c1d6229a49f8fde86ca38ec88f205a7cdf98ff7fdd962cd6"}
Jul 13 21:06:53.889091 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:53.889032    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt" event={"ID":"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a","Type":"ContainerStarted","Data":"cf082b8f5c4bf3610756d0463089030fbdf6953405e48d65fc03f5cb4dab411e"}
Jul 13 21:06:57.916083 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:57.916055    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z5w5k" event={"ID":"3dceb5de-c7b5-4e22-b363-95123272ff81","Type":"ContainerStarted","Data":"602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8"}
Jul 13 21:06:57.917859 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:57.917830    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt" event={"ID":"d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a","Type":"ContainerStarted","Data":"9e88454f6b186e5e4d49d6b072c92fe4efa12e64ab78c37c18b846baf8f1529e"}
Jul 13 21:06:57.917953 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:57.917891    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:06:57.931292 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:57.931252    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-z5w5k" podStartSLOduration=1.53706253 podStartE2EDuration="5.931237432s" totalImagesPullingTime="4.394174902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:06:57.930769084 +0000 UTC m=+778.868973421" watchObservedRunningTime="2026-07-13 21:06:57.931237432 +0000 UTC m=+778.869441755"
Jul 13 21:06:57.952428 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:57.948059    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt" podStartSLOduration=1.16195838 podStartE2EDuration="5.948038703s" totalImagesPullingTime="4.786080323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:06:57.94578706 +0000 UTC m=+778.883991396" watchObservedRunningTime="2026-07-13 21:06:57.948038703 +0000 UTC m=+778.886243015"
Jul 13 21:06:58.042763 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:58.042727    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-z5w5k"]
Jul 13 21:06:59.927793 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:06:59.927755    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-z5w5k" podUID="3dceb5de-c7b5-4e22-b363-95123272ff81" containerName="authorino" containerID="cri-o://602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8" gracePeriod=30
Jul 13 21:07:00.167252 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.167232    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z5w5k"
Jul 13 21:07:00.274806 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.274725    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w95g5\" (UniqueName: \"kubernetes.io/projected/3dceb5de-c7b5-4e22-b363-95123272ff81-kube-api-access-w95g5\") pod \"3dceb5de-c7b5-4e22-b363-95123272ff81\" (UID: \"3dceb5de-c7b5-4e22-b363-95123272ff81\") "
Jul 13 21:07:00.276768 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.276740    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dceb5de-c7b5-4e22-b363-95123272ff81-kube-api-access-w95g5" (OuterVolumeSpecName: "kube-api-access-w95g5") pod "3dceb5de-c7b5-4e22-b363-95123272ff81" (UID: "3dceb5de-c7b5-4e22-b363-95123272ff81"). InnerVolumeSpecName "kube-api-access-w95g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:07:00.375994 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.375968    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w95g5\" (UniqueName: \"kubernetes.io/projected/3dceb5de-c7b5-4e22-b363-95123272ff81-kube-api-access-w95g5\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:07:00.933954 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.933924    2556 generic.go:356] "Generic (PLEG): container finished" podID="3dceb5de-c7b5-4e22-b363-95123272ff81" containerID="602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8" exitCode=0
Jul 13 21:07:00.934460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.933966    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z5w5k"
Jul 13 21:07:00.934460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.934017    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z5w5k" event={"ID":"3dceb5de-c7b5-4e22-b363-95123272ff81","Type":"ContainerDied","Data":"602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8"}
Jul 13 21:07:00.934460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.934063    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z5w5k" event={"ID":"3dceb5de-c7b5-4e22-b363-95123272ff81","Type":"ContainerDied","Data":"6b9cb1cd36e02ad7c1d6229a49f8fde86ca38ec88f205a7cdf98ff7fdd962cd6"}
Jul 13 21:07:00.934460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.934087    2556 scope.go:117] "RemoveContainer" containerID="602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8"
Jul 13 21:07:00.943998 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.943983    2556 scope.go:117] "RemoveContainer" containerID="602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8"
Jul 13 21:07:00.944214 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:07:00.944200    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8\": container with ID starting with 602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8 not found: ID does not exist" containerID="602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8"
Jul 13 21:07:00.944300 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.944221    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8"} err="failed to get container status \"602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8\": rpc error: code = NotFound desc = could not find container \"602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8\": container with ID starting with 602bee85f59c78b52cea7d2a33c0de277494fecd0b752af2db7fc61d538e56f8 not found: ID does not exist"
Jul 13 21:07:00.956141 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.956122    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-z5w5k"]
Jul 13 21:07:00.960310 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:00.960292    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-z5w5k"]
Jul 13 21:07:01.602080 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:01.602049    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dceb5de-c7b5-4e22-b363-95123272ff81" path="/var/lib/kubelet/pods/3dceb5de-c7b5-4e22-b363-95123272ff81/volumes"
Jul 13 21:07:08.924066 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:07:08.924036    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"
Jul 13 21:08:05.960227 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:08:05.960190    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:08:26.560093 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:08:26.560059    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:08:40.038787 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:08:40.038750    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:08:43.252268 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:08:43.252231    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:08:59.530866 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:08:59.530779    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:08:59.532292 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:08:59.532274    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:08:59.944015 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:08:59.943982    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:09:06.457180 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:06.457142    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:09:10.735069 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.735034    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"]
Jul 13 21:09:10.735569 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.735414    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dceb5de-c7b5-4e22-b363-95123272ff81" containerName="authorino"
Jul 13 21:09:10.735569 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.735427    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dceb5de-c7b5-4e22-b363-95123272ff81" containerName="authorino"
Jul 13 21:09:10.735569 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.735499    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dceb5de-c7b5-4e22-b363-95123272ff81" containerName="authorino"
Jul 13 21:09:10.737600 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.737579    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.741402 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.741380    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 13 21:09:10.741755 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.741732    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 21:09:10.741860 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.741732    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fb5tw\""
Jul 13 21:09:10.741922 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.741910    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 21:09:10.751737 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.751713    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"]
Jul 13 21:09:10.811174 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.811144    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.811174 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.811176    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.811487 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.811203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.811487 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.811250    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.811487 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.811281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.811487 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.811355    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brl2l\" (UniqueName: \"kubernetes.io/projected/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-kube-api-access-brl2l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.912595 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.912554    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brl2l\" (UniqueName: \"kubernetes.io/projected/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-kube-api-access-brl2l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.912783 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.912643    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.912783 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.912669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.912783 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.912719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.912783 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.912742    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.912783 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.912777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.913179 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.913152    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.913179 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.913149    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.913365 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.913192    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.915052 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.915033    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.915258 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.915242    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:10.920507 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:10.920485    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brl2l\" (UniqueName: \"kubernetes.io/projected/01b0f5bd-b744-4f07-b2d3-e5b8354f252c-kube-api-access-brl2l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m\" (UID: \"01b0f5bd-b744-4f07-b2d3-e5b8354f252c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:11.047990 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:11.047898    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:11.181497 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:11.181473    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"]
Jul 13 21:09:11.183458 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:09:11.183430    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01b0f5bd_b744_4f07_b2d3_e5b8354f252c.slice/crio-14244763a66856d45bdbd27052770dee7d26c4541d59437234bf5d4cd5f1abaa WatchSource:0}: Error finding container 14244763a66856d45bdbd27052770dee7d26c4541d59437234bf5d4cd5f1abaa: Status 404 returned error can't find the container with id 14244763a66856d45bdbd27052770dee7d26c4541d59437234bf5d4cd5f1abaa
Jul 13 21:09:11.648524 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:11.648468    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m" event={"ID":"01b0f5bd-b744-4f07-b2d3-e5b8354f252c","Type":"ContainerStarted","Data":"14244763a66856d45bdbd27052770dee7d26c4541d59437234bf5d4cd5f1abaa"}
Jul 13 21:09:12.655537 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:12.655492    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m" event={"ID":"01b0f5bd-b744-4f07-b2d3-e5b8354f252c","Type":"ContainerStarted","Data":"aed3cb02cfb291a034dd221b2e0eaa1db1c4abc1837c2e232ed41b8b191bcae0"}
Jul 13 21:09:12.675247 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:12.675199    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m" podStartSLOduration=1.641857774 podStartE2EDuration="2.675183222s" totalImagesPullingTime="1.033325448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:09:12.67210126 +0000 UTC m=+913.610305591" watchObservedRunningTime="2026-07-13 21:09:12.675183222 +0000 UTC m=+913.613387511"
Jul 13 21:09:13.844377 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:13.844330    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:09:21.048803 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:21.048764    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:21.049210 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:21.049015    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:21.061582 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:21.061559    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:09:21.712785 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:09:21.712757    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m"
Jul 13 21:10:08.578894 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.578863    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f7b5d5cfc-449kl"]
Jul 13 21:10:08.582292 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.582275    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:08.585683 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.585659    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 13 21:10:08.585806 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.585688    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7msf9\""
Jul 13 21:10:08.596937 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.596908    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f7b5d5cfc-449kl"]
Jul 13 21:10:08.663754 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.663721    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21efc3d-4ebc-472b-a4d7-5546b51ad755-tls-cert\") pod \"authorino-5f7b5d5cfc-449kl\" (UID: \"c21efc3d-4ebc-472b-a4d7-5546b51ad755\") " pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:08.663754 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.663759    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2znc\" (UniqueName: \"kubernetes.io/projected/c21efc3d-4ebc-472b-a4d7-5546b51ad755-kube-api-access-c2znc\") pod \"authorino-5f7b5d5cfc-449kl\" (UID: \"c21efc3d-4ebc-472b-a4d7-5546b51ad755\") " pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:08.764660 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.764623    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21efc3d-4ebc-472b-a4d7-5546b51ad755-tls-cert\") pod \"authorino-5f7b5d5cfc-449kl\" (UID: \"c21efc3d-4ebc-472b-a4d7-5546b51ad755\") " pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:08.764660 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.764665    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2znc\" (UniqueName: \"kubernetes.io/projected/c21efc3d-4ebc-472b-a4d7-5546b51ad755-kube-api-access-c2znc\") pod \"authorino-5f7b5d5cfc-449kl\" (UID: \"c21efc3d-4ebc-472b-a4d7-5546b51ad755\") " pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:08.766991 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.766971    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21efc3d-4ebc-472b-a4d7-5546b51ad755-tls-cert\") pod \"authorino-5f7b5d5cfc-449kl\" (UID: \"c21efc3d-4ebc-472b-a4d7-5546b51ad755\") " pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:08.772670 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.772645    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2znc\" (UniqueName: \"kubernetes.io/projected/c21efc3d-4ebc-472b-a4d7-5546b51ad755-kube-api-access-c2znc\") pod \"authorino-5f7b5d5cfc-449kl\" (UID: \"c21efc3d-4ebc-472b-a4d7-5546b51ad755\") " pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:08.894637 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:08.894608    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f7b5d5cfc-449kl"
Jul 13 21:10:09.043362 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:09.043316    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f7b5d5cfc-449kl"]
Jul 13 21:10:09.045424 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:10:09.045400    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc21efc3d_4ebc_472b_a4d7_5546b51ad755.slice/crio-75ee1ba1351f11e8f1f065f9bc13c54e61a2bd63197b72efdfd10ae1d2519c89 WatchSource:0}: Error finding container 75ee1ba1351f11e8f1f065f9bc13c54e61a2bd63197b72efdfd10ae1d2519c89: Status 404 returned error can't find the container with id 75ee1ba1351f11e8f1f065f9bc13c54e61a2bd63197b72efdfd10ae1d2519c89
Jul 13 21:10:09.967519 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:09.967480    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f7b5d5cfc-449kl" event={"ID":"c21efc3d-4ebc-472b-a4d7-5546b51ad755","Type":"ContainerStarted","Data":"c8ea1032c42bbb47b1f9c05de8828d3a8230e5e7784bb456cc6dbbf526876621"}
Jul 13 21:10:09.967897 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:09.967530    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f7b5d5cfc-449kl" event={"ID":"c21efc3d-4ebc-472b-a4d7-5546b51ad755","Type":"ContainerStarted","Data":"75ee1ba1351f11e8f1f065f9bc13c54e61a2bd63197b72efdfd10ae1d2519c89"}
Jul 13 21:10:09.985733 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:09.985682    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5f7b5d5cfc-449kl" podStartSLOduration=1.264673086 podStartE2EDuration="1.985668783s" totalImagesPullingTime="720.995697ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:10:09.983075743 +0000 UTC m=+970.921280066" watchObservedRunningTime="2026-07-13 21:10:09.985668783 +0000 UTC m=+970.923873093"
Jul 13 21:10:38.657359 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:38.657266    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:10:48.946186 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:10:48.946151    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:11:00.049501 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:11:00.049471    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:11:08.741083 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:11:08.741044    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:11:20.041186 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:11:20.041152    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:11:47.846390 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:11:47.846350    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:12:02.943147 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:12:02.943067    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:12:35.948531 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:12:35.948491    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:12:45.751087 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:12:45.751050    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:12:54.640406 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:12:54.640374    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:13:05.748238 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:13:05.748202    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:13:16.472784 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:13:16.472749    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:13:26.250959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:13:26.250878    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:13:38.051151 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:13:38.051111    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:13:47.942644 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:13:47.942612    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:13:59.567568 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:13:59.567534    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:13:59.571329 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:13:59.571307    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:14:10.550547 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:14:10.550516    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:14:19.454991 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:14:19.454958    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:14:43.054835 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:14:43.054803    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:14:53.539962 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:14:53.539882    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:15:09.556874 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:15:09.556845    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:15:18.860398 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:15:18.860360    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:15:28.247814 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:15:28.247781    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:15:37.346980 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:15:37.346948    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:15:45.043959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:15:45.043926    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:17:34.551145 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:17:34.551109    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:17:43.546222 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:17:43.546188    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:17:56.347026 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:17:56.346942    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:18:08.554371 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:08.554309    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:18:17.843885 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:17.843856    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:18:30.157350 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:30.157301    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:18:38.444402 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:38.444368    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:18:49.241727 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:49.241695    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:18:59.608172 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:59.608144    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:18:59.614061 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:59.614034    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:18:59.740380 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:18:59.740334    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:19:08.544473 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:19:08.544439    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:19:18.038165 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:19:18.038130    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:19:27.145837 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:19:27.145746    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:19:45.645282 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:19:45.645250    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:19:55.452266 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:19:55.452233    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:20:05.054756 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:20:05.054721    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:20:16.245569 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:20:16.245537    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:20:25.348869 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:20:25.348829    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:20:36.654011 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:20:36.653974    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:20:47.066609 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:20:47.066575    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:20:55.944500 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:20:55.944410    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:21:06.451432 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:21:06.451395    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:21:15.652022 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:21:15.651990    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:21:25.242710 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:21:25.242675    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:21:39.344574 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:21:39.344539    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:21:53.741777 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:21:53.741743    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:22:03.760609 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:22:03.760573    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:22:13.852178 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:22:13.852142    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:22:24.937931 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:22:24.937847    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:22:41.922943 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:22:41.922908    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:22:54.056209 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:22:54.056172    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:23:03.346937 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:23:03.346898    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:23:16.945810 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:23:16.945778    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:23:25.648680 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:23:25.648646    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:23:35.156506 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:23:35.156474    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:23:48.843813 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:23:48.843778    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:23:59.645038 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:23:59.645013    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:23:59.650233 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:23:59.650211    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:24:42.604159 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.604118    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"]
Jul 13 21:24:42.607706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.607687    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"
Jul 13 21:24:42.610149 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.610124    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:24:42.610258 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.610241    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rnndk\""
Jul 13 21:24:42.615988 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.615425    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"]
Jul 13 21:24:42.651149 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.651114    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d9de8c3-fcc6-4664-ab88-160cc1e5dac0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm\" (UID: \"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"
Jul 13 21:24:42.752074 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.752041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d9de8c3-fcc6-4664-ab88-160cc1e5dac0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm\" (UID: \"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"
Jul 13 21:24:42.752752 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.752733    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d9de8c3-fcc6-4664-ab88-160cc1e5dac0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm\" (UID: \"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"
Jul 13 21:24:42.919123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:42.919087    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"
Jul 13 21:24:43.052062 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:43.052033    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"]
Jul 13 21:24:43.054209 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:24:43.054184    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9de8c3_fcc6_4664_ab88_160cc1e5dac0.slice/crio-55b39279ab6af72e7c068c9f858fd8e01ebe2b12c5f9be0c3c04d851a5fa1b42 WatchSource:0}: Error finding container 55b39279ab6af72e7c068c9f858fd8e01ebe2b12c5f9be0c3c04d851a5fa1b42: Status 404 returned error can't find the container with id 55b39279ab6af72e7c068c9f858fd8e01ebe2b12c5f9be0c3c04d851a5fa1b42
Jul 13 21:24:43.056106 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:43.056091    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:24:43.763969 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:43.763934    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm" event={"ID":"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0","Type":"ContainerStarted","Data":"55b39279ab6af72e7c068c9f858fd8e01ebe2b12c5f9be0c3c04d851a5fa1b42"}
Jul 13 21:24:45.778817 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:45.778782    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm" event={"ID":"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0","Type":"ContainerStarted","Data":"cab5ffe757e37ebb396fc17e2e3aac819861d212598ce9485aceacf01c0d309a"}
Jul 13 21:24:45.795744 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:45.795694    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm" podStartSLOduration=1.5834097329999999 podStartE2EDuration="3.795678056s" totalImagesPullingTime="2.212268323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:24:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:24:45.793655666 +0000 UTC m=+1846.731859979" watchObservedRunningTime="2026-07-13 21:24:45.795678056 +0000 UTC m=+1846.733882378"
Jul 13 21:24:46.786274 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:46.786237    2556 generic.go:356] "Generic (PLEG): container finished" podID="9d9de8c3-fcc6-4664-ab88-160cc1e5dac0" containerID="cab5ffe757e37ebb396fc17e2e3aac819861d212598ce9485aceacf01c0d309a" exitCode=0
Jul 13 21:24:46.786636 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:46.786314    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm" event={"ID":"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0","Type":"ContainerDied","Data":"cab5ffe757e37ebb396fc17e2e3aac819861d212598ce9485aceacf01c0d309a"}
Jul 13 21:24:47.925794 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:47.925767    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"
Jul 13 21:24:47.998400 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:47.998367    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d9de8c3-fcc6-4664-ab88-160cc1e5dac0-maas-api-service-ca\") pod \"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0\" (UID: \"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0\") "
Jul 13 21:24:47.998721 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:47.998693    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d9de8c3-fcc6-4664-ab88-160cc1e5dac0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d9de8c3-fcc6-4664-ab88-160cc1e5dac0" (UID: "9d9de8c3-fcc6-4664-ab88-160cc1e5dac0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:24:48.100077 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:48.099974    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d9de8c3-fcc6-4664-ab88-160cc1e5dac0-maas-api-service-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:24:48.802850 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:48.802814    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm"
Jul 13 21:24:48.803038 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:48.802845    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm" event={"ID":"9d9de8c3-fcc6-4664-ab88-160cc1e5dac0","Type":"ContainerDied","Data":"55b39279ab6af72e7c068c9f858fd8e01ebe2b12c5f9be0c3c04d851a5fa1b42"}
Jul 13 21:24:48.803038 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:48.802927    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55b39279ab6af72e7c068c9f858fd8e01ebe2b12c5f9be0c3c04d851a5fa1b42"
Jul 13 21:24:56.237299 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:24:56.237259    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:25:17.451262 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:25:17.451223    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:25:27.553507 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:25:27.553467    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:25:43.838864 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:25:43.838826    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:26:59.349858 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:26:59.349774    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:27:16.243996 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:27:16.243962    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:28:21.739160 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:28:21.739078    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:28:59.684387 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:28:59.684358    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:28:59.692887 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:28:59.692863    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:29:56.955253 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:29:56.955177    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:30:06.248653 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:30:06.248616    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:30:17.846448 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:30:17.846410    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:30:37.046736 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:30:37.046704    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:31:18.948351 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:18.948306    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:31:24.739943 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.739862    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"]
Jul 13 21:31:24.740324 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.740310    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d9de8c3-fcc6-4664-ab88-160cc1e5dac0" containerName="revoke-keys"
Jul 13 21:31:24.740431 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.740325    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d9de8c3-fcc6-4664-ab88-160cc1e5dac0" containerName="revoke-keys"
Jul 13 21:31:24.740477 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.740447    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d9de8c3-fcc6-4664-ab88-160cc1e5dac0" containerName="revoke-keys"
Jul 13 21:31:24.743700 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.743685    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.747511 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.747490    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-11bd6c7a\"/\"kube-root-ca.crt\""
Jul 13 21:31:24.747631 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.747519    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-11bd6c7a\"/\"default-dockercfg-s9fjb\""
Jul 13 21:31:24.747631 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.747528    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-11bd6c7a\"/\"openshift-service-ca.crt\""
Jul 13 21:31:24.747631 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.747528    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-11bd6c7a\"/\"e2e-lifecycle-model-11bd6c7a-kserve-self-signed-certs\""
Jul 13 21:31:24.757319 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.757298    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"]
Jul 13 21:31:24.832153 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.832113    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-dshm\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.832322 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.832211    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-home\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.832322 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.832248    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-tmp-dir\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.832322 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.832286    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-model-cache\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.832322 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.832302    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xqnh\" (UniqueName: \"kubernetes.io/projected/ecbce1bd-6c5c-4688-b206-c12530659294-kube-api-access-5xqnh\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.832322 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.832320    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbce1bd-6c5c-4688-b206-c12530659294-tls-certs\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.933718 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.933686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-dshm\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.933886 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.933744    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-home\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.933886 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.933761    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-tmp-dir\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.933886 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.933792    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-model-cache\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.933886 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.933815    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xqnh\" (UniqueName: \"kubernetes.io/projected/ecbce1bd-6c5c-4688-b206-c12530659294-kube-api-access-5xqnh\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.933886 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.933832    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbce1bd-6c5c-4688-b206-c12530659294-tls-certs\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.934195 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.934150    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-home\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.934273 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.934227    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-model-cache\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.934325 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.934270    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-tmp-dir\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.936121 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.936090    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-dshm\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.936308 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.936293    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbce1bd-6c5c-4688-b206-c12530659294-tls-certs\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:24.942316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:24.942290    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xqnh\" (UniqueName: \"kubernetes.io/projected/ecbce1bd-6c5c-4688-b206-c12530659294-kube-api-access-5xqnh\") pod \"e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") " pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:25.052760 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:25.052646    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:25.196652 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:25.191679    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"]
Jul 13 21:31:25.196652 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:31:25.195061    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecbce1bd_6c5c_4688_b206_c12530659294.slice/crio-9d812b3d083f05ef8ff33044e14ddfe8133ce8d5fa674ca37fef351f0afe8163 WatchSource:0}: Error finding container 9d812b3d083f05ef8ff33044e14ddfe8133ce8d5fa674ca37fef351f0afe8163: Status 404 returned error can't find the container with id 9d812b3d083f05ef8ff33044e14ddfe8133ce8d5fa674ca37fef351f0afe8163
Jul 13 21:31:25.948488 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:25.948452    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb" event={"ID":"ecbce1bd-6c5c-4688-b206-c12530659294","Type":"ContainerStarted","Data":"f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c"}
Jul 13 21:31:25.948871 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:25.948499    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb" event={"ID":"ecbce1bd-6c5c-4688-b206-c12530659294","Type":"ContainerStarted","Data":"9d812b3d083f05ef8ff33044e14ddfe8133ce8d5fa674ca37fef351f0afe8163"}
Jul 13 21:31:25.966074 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:25.966025    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb" podStartSLOduration=1.9660103389999999 podStartE2EDuration="1.966010339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:31:25.965221654 +0000 UTC m=+2246.903425966" watchObservedRunningTime="2026-07-13 21:31:25.966010339 +0000 UTC m=+2246.904214650"
Jul 13 21:31:30.458117 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:30.458088    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:31:35.053681 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:35.053646    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:35.054114 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:35.053693    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:35.066783 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:35.066757    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:36.015204 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:36.015177    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:31:49.742914 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:31:49.742879    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:32:09.247123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:09.247084    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:32:15.349823 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:15.349786    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"]
Jul 13 21:32:30.370639 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.370596    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb" podUID="ecbce1bd-6c5c-4688-b206-c12530659294" containerName="main" containerID="cri-o://f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c" gracePeriod=45
Jul 13 21:32:30.619549 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.619524    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:32:30.722169 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722076    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-tmp-dir\") pod \"ecbce1bd-6c5c-4688-b206-c12530659294\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") "
Jul 13 21:32:30.722169 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722125    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xqnh\" (UniqueName: \"kubernetes.io/projected/ecbce1bd-6c5c-4688-b206-c12530659294-kube-api-access-5xqnh\") pod \"ecbce1bd-6c5c-4688-b206-c12530659294\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") "
Jul 13 21:32:30.722169 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722161    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-dshm\") pod \"ecbce1bd-6c5c-4688-b206-c12530659294\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") "
Jul 13 21:32:30.722494 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722189    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-home\") pod \"ecbce1bd-6c5c-4688-b206-c12530659294\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") "
Jul 13 21:32:30.722494 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722223    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-model-cache\") pod \"ecbce1bd-6c5c-4688-b206-c12530659294\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") "
Jul 13 21:32:30.722494 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722245    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbce1bd-6c5c-4688-b206-c12530659294-tls-certs\") pod \"ecbce1bd-6c5c-4688-b206-c12530659294\" (UID: \"ecbce1bd-6c5c-4688-b206-c12530659294\") "
Jul 13 21:32:30.722652 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722548    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-home" (OuterVolumeSpecName: "home") pod "ecbce1bd-6c5c-4688-b206-c12530659294" (UID: "ecbce1bd-6c5c-4688-b206-c12530659294"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:32:30.722652 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722561    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-model-cache" (OuterVolumeSpecName: "model-cache") pod "ecbce1bd-6c5c-4688-b206-c12530659294" (UID: "ecbce1bd-6c5c-4688-b206-c12530659294"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:32:30.722652 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.722596    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ecbce1bd-6c5c-4688-b206-c12530659294" (UID: "ecbce1bd-6c5c-4688-b206-c12530659294"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:32:30.724508 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.724481    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecbce1bd-6c5c-4688-b206-c12530659294-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ecbce1bd-6c5c-4688-b206-c12530659294" (UID: "ecbce1bd-6c5c-4688-b206-c12530659294"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:32:30.724622 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.724536    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-dshm" (OuterVolumeSpecName: "dshm") pod "ecbce1bd-6c5c-4688-b206-c12530659294" (UID: "ecbce1bd-6c5c-4688-b206-c12530659294"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:32:30.724622 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.724550    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecbce1bd-6c5c-4688-b206-c12530659294-kube-api-access-5xqnh" (OuterVolumeSpecName: "kube-api-access-5xqnh") pod "ecbce1bd-6c5c-4688-b206-c12530659294" (UID: "ecbce1bd-6c5c-4688-b206-c12530659294"). InnerVolumeSpecName "kube-api-access-5xqnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:32:30.823763 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.823723    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xqnh\" (UniqueName: \"kubernetes.io/projected/ecbce1bd-6c5c-4688-b206-c12530659294-kube-api-access-5xqnh\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:32:30.823763 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.823756    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:32:30.823763 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.823768    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:32:30.824002 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.823779    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:32:30.824002 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.823792    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbce1bd-6c5c-4688-b206-c12530659294-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:32:30.824002 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:30.823804    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbce1bd-6c5c-4688-b206-c12530659294-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:32:31.309070 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.309034    2556 generic.go:356] "Generic (PLEG): container finished" podID="ecbce1bd-6c5c-4688-b206-c12530659294" containerID="f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c" exitCode=0
Jul 13 21:32:31.309070 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.309074    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"
Jul 13 21:32:31.309291 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.309078    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb" event={"ID":"ecbce1bd-6c5c-4688-b206-c12530659294","Type":"ContainerDied","Data":"f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c"}
Jul 13 21:32:31.309291 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.309113    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb" event={"ID":"ecbce1bd-6c5c-4688-b206-c12530659294","Type":"ContainerDied","Data":"9d812b3d083f05ef8ff33044e14ddfe8133ce8d5fa674ca37fef351f0afe8163"}
Jul 13 21:32:31.309291 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.309131    2556 scope.go:117] "RemoveContainer" containerID="f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c"
Jul 13 21:32:31.318574 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.318552    2556 scope.go:117] "RemoveContainer" containerID="f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c"
Jul 13 21:32:31.318829 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:32:31.318811    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c\": container with ID starting with f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c not found: ID does not exist" containerID="f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c"
Jul 13 21:32:31.318879 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.318838    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c"} err="failed to get container status \"f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c\": rpc error: code = NotFound desc = could not find container \"f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c\": container with ID starting with f7aa9bf56a86a013d4168614da14e753d7c35c9aefbc1ea82e7b871cf7c9411c not found: ID does not exist"
Jul 13 21:32:31.330973 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.330951    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"]
Jul 13 21:32:31.335369 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.335348    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-11bd6c7a/e2e-lifecycle-model-11bd6c7a-kserve-7bc5bc49d7-s7zkb"]
Jul 13 21:32:31.603265 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:32:31.603185    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecbce1bd-6c5c-4688-b206-c12530659294" path="/var/lib/kubelet/pods/ecbce1bd-6c5c-4688-b206-c12530659294/volumes"
Jul 13 21:33:26.847323 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:33:26.847281    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:33:39.946628 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:33:39.946595    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:33:59.725462 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:33:59.725430    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:33:59.734277 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:33:59.734252    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:34:28.645212 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:28.645128    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:34:42.948661 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:42.948631    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:34:43.240435 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.240356    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"]
Jul 13 21:34:43.240840 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.240821    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecbce1bd-6c5c-4688-b206-c12530659294" containerName="main"
Jul 13 21:34:43.240932 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.240842    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbce1bd-6c5c-4688-b206-c12530659294" containerName="main"
Jul 13 21:34:43.240992 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.240931    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecbce1bd-6c5c-4688-b206-c12530659294" containerName="main"
Jul 13 21:34:43.244057 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.244039    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.247520 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.247494    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-613d27\"/\"openshift-service-ca.crt\""
Jul 13 21:34:43.247634 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.247498    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-613d27\"/\"test-model-613d27-kserve-self-signed-certs\""
Jul 13 21:34:43.247634 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.247505    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-613d27\"/\"default-dockercfg-qsrzl\""
Jul 13 21:34:43.247634 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.247559    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-613d27\"/\"kube-root-ca.crt\""
Jul 13 21:34:43.256615 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.256594    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"]
Jul 13 21:34:43.352276 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.352243    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-dshm\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.352458 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.352288    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-model-cache\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.352458 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.352401    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tmp-dir\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.352458 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.352435    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-home\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.352568 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.352462    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tls-certs\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.352568 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.352491    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54n5t\" (UniqueName: \"kubernetes.io/projected/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-kube-api-access-54n5t\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.453800 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.453766    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-dshm\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.453927 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.453816    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-model-cache\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.453927 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.453855    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tmp-dir\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.453927 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.453876    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-home\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.453927 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.453892    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tls-certs\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.453927 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.453917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54n5t\" (UniqueName: \"kubernetes.io/projected/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-kube-api-access-54n5t\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.454298 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.454272    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tmp-dir\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.454394 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.454300    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-model-cache\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.454439 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.454404    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-home\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.456195 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.456173    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-dshm\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.456400 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.456382    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tls-certs\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.462017 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.461999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54n5t\" (UniqueName: \"kubernetes.io/projected/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-kube-api-access-54n5t\") pod \"test-model-613d27-kserve-b98d44d8c-s6fdf\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") " pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.553401 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.553297    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:43.687489 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:43.687457    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"]
Jul 13 21:34:43.688935 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:34:43.688904    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c6eeeb3_ac5c_4931_9169_525739d1bdd0.slice/crio-8b40c595e9bef0d43538d0a64fb6391e598c3fa6f955d2a33c5c21882f508b93 WatchSource:0}: Error finding container 8b40c595e9bef0d43538d0a64fb6391e598c3fa6f955d2a33c5c21882f508b93: Status 404 returned error can't find the container with id 8b40c595e9bef0d43538d0a64fb6391e598c3fa6f955d2a33c5c21882f508b93
Jul 13 21:34:44.034827 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:44.034793    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf" event={"ID":"7c6eeeb3-ac5c-4931-9169-525739d1bdd0","Type":"ContainerStarted","Data":"c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb"}
Jul 13 21:34:44.035267 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:44.034836    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf" event={"ID":"7c6eeeb3-ac5c-4931-9169-525739d1bdd0","Type":"ContainerStarted","Data":"8b40c595e9bef0d43538d0a64fb6391e598c3fa6f955d2a33c5c21882f508b93"}
Jul 13 21:34:44.052849 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:44.052797    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf" podStartSLOduration=1.052780345 podStartE2EDuration="1.052780345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:44.051138918 +0000 UTC m=+2444.989343233" watchObservedRunningTime="2026-07-13 21:34:44.052780345 +0000 UTC m=+2444.990984656"
Jul 13 21:34:53.553483 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:53.553450    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:53.554058 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:53.553520    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:53.565538 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:53.565516    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:54.110841 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:54.110817    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:34:55.151929 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:34:55.151890    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:35:14.153828 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.153794    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"]
Jul 13 21:35:14.157139 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.157123    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.161729 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.161710    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2a506c\"/\"default-dockercfg-7458q\""
Jul 13 21:35:14.161865 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.161750    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2a506c\"/\"openshift-service-ca.crt\""
Jul 13 21:35:14.161865 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.161750    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2a506c\"/\"kube-root-ca.crt\""
Jul 13 21:35:14.161865 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.161804    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2a506c\"/\"test-model-2a506c-kserve-self-signed-certs\""
Jul 13 21:35:14.168718 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.168696    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"]
Jul 13 21:35:14.217490 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.217453    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-dshm\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.217661 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.217509    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tmp-dir\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.217661 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.217572    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-home\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.217661 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.217615    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tls-certs\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.217661 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.217647    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-model-cache\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.217834 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.217716    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rt5k\" (UniqueName: \"kubernetes.io/projected/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-kube-api-access-4rt5k\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.318631 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.318592    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tls-certs\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.318835 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.318646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-model-cache\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.318835 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.318686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rt5k\" (UniqueName: \"kubernetes.io/projected/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-kube-api-access-4rt5k\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.318835 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.318747    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-dshm\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.318835 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.318790    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tmp-dir\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.319059 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.318933    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-home\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.319118 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.319081    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-model-cache\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.319175 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.319144    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tmp-dir\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.319265 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.319248    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-home\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.320964 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.320946    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-dshm\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.321140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.321115    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tls-certs\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.326761 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.326738    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rt5k\" (UniqueName: \"kubernetes.io/projected/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-kube-api-access-4rt5k\") pod \"test-model-2a506c-kserve-5c5fd7f875-nhdbx\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") " pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.466891 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.466804    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:14.601867 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:14.601839    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"]
Jul 13 21:35:14.603569 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:35:14.603534    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0ea8357_93cc_4d2c_a19c_bfedfa8545df.slice/crio-f7eeef9f6a1edc4776d5425e48908fe57dbcb35d12bc08f206fb02c4e8ab4f41 WatchSource:0}: Error finding container f7eeef9f6a1edc4776d5425e48908fe57dbcb35d12bc08f206fb02c4e8ab4f41: Status 404 returned error can't find the container with id f7eeef9f6a1edc4776d5425e48908fe57dbcb35d12bc08f206fb02c4e8ab4f41
Jul 13 21:35:15.216528 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:15.216489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx" event={"ID":"e0ea8357-93cc-4d2c-a19c-bfedfa8545df","Type":"ContainerStarted","Data":"1abeff26a8a17019465c1a81b6bc9fa665299cd8a8f1582ee60ea497c65d5c27"}
Jul 13 21:35:15.216528 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:15.216535    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx" event={"ID":"e0ea8357-93cc-4d2c-a19c-bfedfa8545df","Type":"ContainerStarted","Data":"f7eeef9f6a1edc4776d5425e48908fe57dbcb35d12bc08f206fb02c4e8ab4f41"}
Jul 13 21:35:15.268570 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:15.268522    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx" podStartSLOduration=1.268501599 podStartE2EDuration="1.268501599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:35:15.266882148 +0000 UTC m=+2476.205086459" watchObservedRunningTime="2026-07-13 21:35:15.268501599 +0000 UTC m=+2476.206705908"
Jul 13 21:35:17.741125 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:17.741092    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:35:24.467291 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:24.467257    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:24.467879 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:24.467307    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:24.480317 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:24.480288    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:25.280631 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:25.280601    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:31.618251 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:31.618221    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:35:32.119903 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:32.119870    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"]
Jul 13 21:35:33.156488 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:33.156459    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"]
Jul 13 21:35:43.948449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:43.948414    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:35:47.140360 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.140301    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf" podUID="7c6eeeb3-ac5c-4931-9169-525739d1bdd0" containerName="main" containerID="cri-o://c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb" gracePeriod=45
Jul 13 21:35:47.395862 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.395794    2556 generic.go:356] "Generic (PLEG): container finished" podID="7c6eeeb3-ac5c-4931-9169-525739d1bdd0" containerID="c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb" exitCode=0
Jul 13 21:35:47.395862 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.395812    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:35:47.395862 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.395850    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf" event={"ID":"7c6eeeb3-ac5c-4931-9169-525739d1bdd0","Type":"ContainerDied","Data":"c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb"}
Jul 13 21:35:47.396100 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.395883    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf" event={"ID":"7c6eeeb3-ac5c-4931-9169-525739d1bdd0","Type":"ContainerDied","Data":"8b40c595e9bef0d43538d0a64fb6391e598c3fa6f955d2a33c5c21882f508b93"}
Jul 13 21:35:47.396100 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.395900    2556 scope.go:117] "RemoveContainer" containerID="c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb"
Jul 13 21:35:47.404700 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.404682    2556 scope.go:117] "RemoveContainer" containerID="c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb"
Jul 13 21:35:47.404930 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:35:47.404913    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb\": container with ID starting with c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb not found: ID does not exist" containerID="c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb"
Jul 13 21:35:47.404969 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.404939    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb"} err="failed to get container status \"c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb\": rpc error: code = NotFound desc = could not find container \"c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb\": container with ID starting with c17446c30a6084a8b1c435e9238f9582bc60edfe90bf10813ea7d0963ab8cefb not found: ID does not exist"
Jul 13 21:35:47.524947 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.524910    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-dshm\") pod \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") "
Jul 13 21:35:47.525143 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.524994    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-model-cache\") pod \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") "
Jul 13 21:35:47.525143 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525014    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54n5t\" (UniqueName: \"kubernetes.io/projected/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-kube-api-access-54n5t\") pod \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") "
Jul 13 21:35:47.525143 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525084    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tls-certs\") pod \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") "
Jul 13 21:35:47.525143 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525121    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-home\") pod \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") "
Jul 13 21:35:47.525380 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525151    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tmp-dir\") pod \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\" (UID: \"7c6eeeb3-ac5c-4931-9169-525739d1bdd0\") "
Jul 13 21:35:47.525380 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525274    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c6eeeb3-ac5c-4931-9169-525739d1bdd0" (UID: "7c6eeeb3-ac5c-4931-9169-525739d1bdd0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:47.525511 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525411    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-home" (OuterVolumeSpecName: "home") pod "7c6eeeb3-ac5c-4931-9169-525739d1bdd0" (UID: "7c6eeeb3-ac5c-4931-9169-525739d1bdd0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:47.525511 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525499    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:47.525605 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525525    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:47.525605 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.525486    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c6eeeb3-ac5c-4931-9169-525739d1bdd0" (UID: "7c6eeeb3-ac5c-4931-9169-525739d1bdd0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:47.527333 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.527308    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c6eeeb3-ac5c-4931-9169-525739d1bdd0" (UID: "7c6eeeb3-ac5c-4931-9169-525739d1bdd0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:47.527473 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.527369    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-dshm" (OuterVolumeSpecName: "dshm") pod "7c6eeeb3-ac5c-4931-9169-525739d1bdd0" (UID: "7c6eeeb3-ac5c-4931-9169-525739d1bdd0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:47.527473 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.527390    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-kube-api-access-54n5t" (OuterVolumeSpecName: "kube-api-access-54n5t") pod "7c6eeeb3-ac5c-4931-9169-525739d1bdd0" (UID: "7c6eeeb3-ac5c-4931-9169-525739d1bdd0"). InnerVolumeSpecName "kube-api-access-54n5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:35:47.626135 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.626106    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:47.626135 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.626128    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:47.626135 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.626137    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:47.626331 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:47.626145    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54n5t\" (UniqueName: \"kubernetes.io/projected/7c6eeeb3-ac5c-4931-9169-525739d1bdd0-kube-api-access-54n5t\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:48.176560 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.176520    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx" podUID="e0ea8357-93cc-4d2c-a19c-bfedfa8545df" containerName="main" containerID="cri-o://1abeff26a8a17019465c1a81b6bc9fa665299cd8a8f1582ee60ea497c65d5c27" gracePeriod=45
Jul 13 21:35:48.401954 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.401924    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"
Jul 13 21:35:48.403982 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.403949    2556 generic.go:356] "Generic (PLEG): container finished" podID="e0ea8357-93cc-4d2c-a19c-bfedfa8545df" containerID="1abeff26a8a17019465c1a81b6bc9fa665299cd8a8f1582ee60ea497c65d5c27" exitCode=0
Jul 13 21:35:48.404127 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.403988    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx" event={"ID":"e0ea8357-93cc-4d2c-a19c-bfedfa8545df","Type":"ContainerDied","Data":"1abeff26a8a17019465c1a81b6bc9fa665299cd8a8f1582ee60ea497c65d5c27"}
Jul 13 21:35:48.418990 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.418965    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"]
Jul 13 21:35:48.423562 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.423539    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-613d27/test-model-613d27-kserve-b98d44d8c-s6fdf"]
Jul 13 21:35:48.429460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.429413    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:48.533355 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533306    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tls-certs\") pod \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") "
Jul 13 21:35:48.533533 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533384    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-model-cache\") pod \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") "
Jul 13 21:35:48.533533 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533423    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tmp-dir\") pod \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") "
Jul 13 21:35:48.533533 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533467    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-dshm\") pod \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") "
Jul 13 21:35:48.533533 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533496    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rt5k\" (UniqueName: \"kubernetes.io/projected/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-kube-api-access-4rt5k\") pod \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") "
Jul 13 21:35:48.533769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533543    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-home\") pod \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\" (UID: \"e0ea8357-93cc-4d2c-a19c-bfedfa8545df\") "
Jul 13 21:35:48.533769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533692    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0ea8357-93cc-4d2c-a19c-bfedfa8545df" (UID: "e0ea8357-93cc-4d2c-a19c-bfedfa8545df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:48.533769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533708    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0ea8357-93cc-4d2c-a19c-bfedfa8545df" (UID: "e0ea8357-93cc-4d2c-a19c-bfedfa8545df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:48.533891 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533779    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:48.534001 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.533981    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-home" (OuterVolumeSpecName: "home") pod "e0ea8357-93cc-4d2c-a19c-bfedfa8545df" (UID: "e0ea8357-93cc-4d2c-a19c-bfedfa8545df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:48.535660 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.535633    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-dshm" (OuterVolumeSpecName: "dshm") pod "e0ea8357-93cc-4d2c-a19c-bfedfa8545df" (UID: "e0ea8357-93cc-4d2c-a19c-bfedfa8545df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:48.535765 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.535746    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0ea8357-93cc-4d2c-a19c-bfedfa8545df" (UID: "e0ea8357-93cc-4d2c-a19c-bfedfa8545df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:48.535823 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.535769    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-kube-api-access-4rt5k" (OuterVolumeSpecName: "kube-api-access-4rt5k") pod "e0ea8357-93cc-4d2c-a19c-bfedfa8545df" (UID: "e0ea8357-93cc-4d2c-a19c-bfedfa8545df"). InnerVolumeSpecName "kube-api-access-4rt5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:35:48.634899 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.634867    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:48.634899 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.634893    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rt5k\" (UniqueName: \"kubernetes.io/projected/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-kube-api-access-4rt5k\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:48.634899 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.634904    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:48.635117 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.634913    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:48.635117 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:48.634921    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ea8357-93cc-4d2c-a19c-bfedfa8545df-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:35:49.410461 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:49.410425    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx" event={"ID":"e0ea8357-93cc-4d2c-a19c-bfedfa8545df","Type":"ContainerDied","Data":"f7eeef9f6a1edc4776d5425e48908fe57dbcb35d12bc08f206fb02c4e8ab4f41"}
Jul 13 21:35:49.410888 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:49.410482    2556 scope.go:117] "RemoveContainer" containerID="1abeff26a8a17019465c1a81b6bc9fa665299cd8a8f1582ee60ea497c65d5c27"
Jul 13 21:35:49.410888 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:49.410441    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"
Jul 13 21:35:49.431975 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:49.431947    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"]
Jul 13 21:35:49.436600 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:49.436575    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2a506c/test-model-2a506c-kserve-5c5fd7f875-nhdbx"]
Jul 13 21:35:49.603377 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:49.603330    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c6eeeb3-ac5c-4931-9169-525739d1bdd0" path="/var/lib/kubelet/pods/7c6eeeb3-ac5c-4931-9169-525739d1bdd0/volumes"
Jul 13 21:35:49.603743 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:49.603730    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0ea8357-93cc-4d2c-a19c-bfedfa8545df" path="/var/lib/kubelet/pods/e0ea8357-93cc-4d2c-a19c-bfedfa8545df/volumes"
Jul 13 21:35:54.540665 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:54.540585    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:35:59.413698 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.413664    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"]
Jul 13 21:35:59.414140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.414028    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6eeeb3-ac5c-4931-9169-525739d1bdd0" containerName="main"
Jul 13 21:35:59.414140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.414038    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6eeeb3-ac5c-4931-9169-525739d1bdd0" containerName="main"
Jul 13 21:35:59.414140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.414067    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0ea8357-93cc-4d2c-a19c-bfedfa8545df" containerName="main"
Jul 13 21:35:59.414140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.414072    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ea8357-93cc-4d2c-a19c-bfedfa8545df" containerName="main"
Jul 13 21:35:59.414140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.414127    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6eeeb3-ac5c-4931-9169-525739d1bdd0" containerName="main"
Jul 13 21:35:59.414140 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.414135    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0ea8357-93cc-4d2c-a19c-bfedfa8545df" containerName="main"
Jul 13 21:35:59.418672 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.418655    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"
Jul 13 21:35:59.421585 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.421550    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rnndk\""
Jul 13 21:35:59.421714 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.421660    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:35:59.423905 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.423875    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69882e5-858a-45a9-bc11-b7b1711b2fb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8\" (UID: \"d69882e5-858a-45a9-bc11-b7b1711b2fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"
Jul 13 21:35:59.424029 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.423901    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"]
Jul 13 21:35:59.524321 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.524286    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69882e5-858a-45a9-bc11-b7b1711b2fb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8\" (UID: \"d69882e5-858a-45a9-bc11-b7b1711b2fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"
Jul 13 21:35:59.526754 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.526736    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:35:59.535216 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.535192    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69882e5-858a-45a9-bc11-b7b1711b2fb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8\" (UID: \"d69882e5-858a-45a9-bc11-b7b1711b2fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"
Jul 13 21:35:59.732584 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.732504    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rnndk\""
Jul 13 21:35:59.740472 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.740452    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"
Jul 13 21:35:59.875205 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:35:59.875177    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"]
Jul 13 21:35:59.877381 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:35:59.877353    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69882e5_858a_45a9_bc11_b7b1711b2fb8.slice/crio-8ac4d6d3a2583bd0e08281ddb2cc4c04ec209fc1eeb80341a3b5f17331fa11df WatchSource:0}: Error finding container 8ac4d6d3a2583bd0e08281ddb2cc4c04ec209fc1eeb80341a3b5f17331fa11df: Status 404 returned error can't find the container with id 8ac4d6d3a2583bd0e08281ddb2cc4c04ec209fc1eeb80341a3b5f17331fa11df
Jul 13 21:36:00.475005 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:00.474975    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8" event={"ID":"d69882e5-858a-45a9-bc11-b7b1711b2fb8","Type":"ContainerStarted","Data":"3f0136f8f3f62c81cde0dce45037dca7020c1e695e7806d9814ab2ca74e0bfbe"}
Jul 13 21:36:00.475382 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:00.475020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8" event={"ID":"d69882e5-858a-45a9-bc11-b7b1711b2fb8","Type":"ContainerStarted","Data":"8ac4d6d3a2583bd0e08281ddb2cc4c04ec209fc1eeb80341a3b5f17331fa11df"}
Jul 13 21:36:00.493402 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:00.493327    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8" podStartSLOduration=1.493311844 podStartE2EDuration="1.493311844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:36:00.492579737 +0000 UTC m=+2521.430784060" watchObservedRunningTime="2026-07-13 21:36:00.493311844 +0000 UTC m=+2521.431516155"
Jul 13 21:36:01.482196 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:01.482167    2556 generic.go:356] "Generic (PLEG): container finished" podID="d69882e5-858a-45a9-bc11-b7b1711b2fb8" containerID="3f0136f8f3f62c81cde0dce45037dca7020c1e695e7806d9814ab2ca74e0bfbe" exitCode=0
Jul 13 21:36:01.482638 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:01.482256    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8" event={"ID":"d69882e5-858a-45a9-bc11-b7b1711b2fb8","Type":"ContainerDied","Data":"3f0136f8f3f62c81cde0dce45037dca7020c1e695e7806d9814ab2ca74e0bfbe"}
Jul 13 21:36:02.620760 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:02.620736    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"
Jul 13 21:36:02.647486 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:02.647459    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69882e5-858a-45a9-bc11-b7b1711b2fb8-maas-api-service-ca\") pod \"d69882e5-858a-45a9-bc11-b7b1711b2fb8\" (UID: \"d69882e5-858a-45a9-bc11-b7b1711b2fb8\") "
Jul 13 21:36:02.647796 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:02.647773    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d69882e5-858a-45a9-bc11-b7b1711b2fb8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d69882e5-858a-45a9-bc11-b7b1711b2fb8" (UID: "d69882e5-858a-45a9-bc11-b7b1711b2fb8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:36:02.748797 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:02.748696    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d69882e5-858a-45a9-bc11-b7b1711b2fb8-maas-api-service-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:36:03.494518 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:03.494490    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8"
Jul 13 21:36:03.494669 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:03.494489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8" event={"ID":"d69882e5-858a-45a9-bc11-b7b1711b2fb8","Type":"ContainerDied","Data":"8ac4d6d3a2583bd0e08281ddb2cc4c04ec209fc1eeb80341a3b5f17331fa11df"}
Jul 13 21:36:03.494669 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:03.494609    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ac4d6d3a2583bd0e08281ddb2cc4c04ec209fc1eeb80341a3b5f17331fa11df"
Jul 13 21:36:12.745319 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:12.745283    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:36:34.951400 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:34.951357    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:36:52.650478 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:36:52.650444    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:37:04.445965 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:37:04.445921    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:37:29.642728 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:37:29.642648    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:37:51.547539 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:37:51.547506    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:38:06.358004 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:06.357969    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:38:08.057860 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.057824    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"]
Jul 13 21:38:08.058225 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.058218    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69882e5-858a-45a9-bc11-b7b1711b2fb8" containerName="revoke-keys"
Jul 13 21:38:08.058267 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.058228    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69882e5-858a-45a9-bc11-b7b1711b2fb8" containerName="revoke-keys"
Jul 13 21:38:08.058303 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.058298    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69882e5-858a-45a9-bc11-b7b1711b2fb8" containerName="revoke-keys"
Jul 13 21:38:08.061417 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.061402    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.064216 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.064190    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0e15bb\"/\"kube-root-ca.crt\""
Jul 13 21:38:08.065141 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.065124    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e15bb\"/\"auth-test-model-0e15bb-kserve-self-signed-certs\""
Jul 13 21:38:08.065141 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.065136    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e15bb\"/\"default-dockercfg-ttbhl\""
Jul 13 21:38:08.065282 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.065134    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0e15bb\"/\"openshift-service-ca.crt\""
Jul 13 21:38:08.070157 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.070140    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"]
Jul 13 21:38:08.167643 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.167603    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-dshm\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.167821 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.167663    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tls-certs\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.167821 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.167737    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-home\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.167821 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.167775    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tmp-dir\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.167821 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.167805    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-model-cache\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.167989 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.167862    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-584t2\" (UniqueName: \"kubernetes.io/projected/c6d20229-42a9-465c-a3dc-b050f83b8ba8-kube-api-access-584t2\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.268699 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.268648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tls-certs\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.268901 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.268710    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-home\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.268901 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.268746    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tmp-dir\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.268901 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.268780    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-model-cache\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.268901 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.268843    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-584t2\" (UniqueName: \"kubernetes.io/projected/c6d20229-42a9-465c-a3dc-b050f83b8ba8-kube-api-access-584t2\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.268901 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.268892    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-dshm\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.269184 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.269086    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-home\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.269184 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.269123    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tmp-dir\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.269184 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.269169    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-model-cache\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.270917 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.270896    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-dshm\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.271259 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.271243    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tls-certs\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.276423 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.276407    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-584t2\" (UniqueName: \"kubernetes.io/projected/c6d20229-42a9-465c-a3dc-b050f83b8ba8-kube-api-access-584t2\") pod \"auth-test-model-0e15bb-kserve-749947979-cblct\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") " pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.372670 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.372634    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:08.509372 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:08.509331    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"]
Jul 13 21:38:08.510952 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:38:08.510927    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6d20229_42a9_465c_a3dc_b050f83b8ba8.slice/crio-0f74ed7e5bbcc0f22f79adf23b0e7212a1faa72c48b8fc43e710ca0630e8dfe1 WatchSource:0}: Error finding container 0f74ed7e5bbcc0f22f79adf23b0e7212a1faa72c48b8fc43e710ca0630e8dfe1: Status 404 returned error can't find the container with id 0f74ed7e5bbcc0f22f79adf23b0e7212a1faa72c48b8fc43e710ca0630e8dfe1
Jul 13 21:38:09.194191 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:09.194157    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct" event={"ID":"c6d20229-42a9-465c-a3dc-b050f83b8ba8","Type":"ContainerStarted","Data":"29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce"}
Jul 13 21:38:09.194607 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:09.194201    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct" event={"ID":"c6d20229-42a9-465c-a3dc-b050f83b8ba8","Type":"ContainerStarted","Data":"0f74ed7e5bbcc0f22f79adf23b0e7212a1faa72c48b8fc43e710ca0630e8dfe1"}
Jul 13 21:38:09.212050 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:09.212003    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct" podStartSLOduration=1.211988152 podStartE2EDuration="1.211988152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:09.210832106 +0000 UTC m=+2650.149036419" watchObservedRunningTime="2026-07-13 21:38:09.211988152 +0000 UTC m=+2650.150192657"
Jul 13 21:38:18.373760 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:18.373717    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:18.374169 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:18.373967    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:18.386468 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:18.386438    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:19.266370 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:19.262291    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:38:20.239626 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:20.239588    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:38:59.763926 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:59.763836    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:38:59.772873 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:38:59.772849    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:39:06.548292 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.548261    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"]
Jul 13 21:39:06.551780 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.551765    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.555246 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.555223    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"auth-test-model-18e186-kserve-self-signed-certs\""
Jul 13 21:39:06.555246 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.555233    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"default-dockercfg-chk48\""
Jul 13 21:39:06.555449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.555251    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"openshift-service-ca.crt\""
Jul 13 21:39:06.555449 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.555288    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"kube-root-ca.crt\""
Jul 13 21:39:06.559767 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.559747    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"]
Jul 13 21:39:06.691479 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.691447    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tls-certs\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.691659 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.691492    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-home\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.691659 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.691510    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-model-cache\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.691659 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.691587    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-dshm\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.691659 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.691639    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tmp-dir\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.691805 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.691691    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztg6j\" (UniqueName: \"kubernetes.io/projected/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-kube-api-access-ztg6j\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.792612 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.792580    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tls-certs\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.792806 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.792777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-home\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.792806 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.792802    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-model-cache\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.792942 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.792842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-dshm\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.793020 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.792994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tmp-dir\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.793092 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.793073    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztg6j\" (UniqueName: \"kubernetes.io/projected/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-kube-api-access-ztg6j\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.793204 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.793163    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-model-cache\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.793318 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.793214    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-home\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.793392 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.793309    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tmp-dir\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.794849 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.794822    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-dshm\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.794935 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.794894    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tls-certs\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.802782 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.802719    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztg6j\" (UniqueName: \"kubernetes.io/projected/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-kube-api-access-ztg6j\") pod \"auth-test-model-18e186-kserve-65cdd9dfff-jhpwp\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") " pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.861708 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.861675    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:06.993294 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:06.993269    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"]
Jul 13 21:39:06.995423 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:39:06.995398    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a8eb34c_928b_4860_adb5_bfdc5e78b2f0.slice/crio-2ddb9f926de992a9033be0a8f78ab278652f24dd633939bd17af639057bccc14 WatchSource:0}: Error finding container 2ddb9f926de992a9033be0a8f78ab278652f24dd633939bd17af639057bccc14: Status 404 returned error can't find the container with id 2ddb9f926de992a9033be0a8f78ab278652f24dd633939bd17af639057bccc14
Jul 13 21:39:07.511639 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:07.511607    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp" event={"ID":"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0","Type":"ContainerStarted","Data":"34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348"}
Jul 13 21:39:07.511847 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:07.511652    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp" event={"ID":"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0","Type":"ContainerStarted","Data":"2ddb9f926de992a9033be0a8f78ab278652f24dd633939bd17af639057bccc14"}
Jul 13 21:39:07.534304 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:07.534252    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp" podStartSLOduration=1.534234808 podStartE2EDuration="1.534234808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:39:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:39:07.531443434 +0000 UTC m=+2708.469647783" watchObservedRunningTime="2026-07-13 21:39:07.534234808 +0000 UTC m=+2708.472439160"
Jul 13 21:39:10.540887 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:10.540853    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:39:16.862421 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:16.862385    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:16.862816 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:16.862571    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:16.875092 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:16.875062    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:17.579956 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:17.579921    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:39:25.750624 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:25.750576    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:39:45.254413 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:39:45.254376    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:40:02.752677 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:40:02.752640    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:40:20.245101 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:40:20.245062    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:40:37.535141 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:40:37.535062    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:40:52.046403 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:40:52.046367    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:41:09.645925 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:09.645888    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:41:12.756132 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:12.756099    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"]
Jul 13 21:41:13.822198 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:13.822164    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"]
Jul 13 21:41:24.845439 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.845401    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"]
Jul 13 21:41:24.849508 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.849491    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:24.851925 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.851905    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e15bb\"/\"sub-test-model-0e15bb-kserve-self-signed-certs\""
Jul 13 21:41:24.856748 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.856490    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"]
Jul 13 21:41:24.934054 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.934017    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-dshm\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:24.934270 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.934075    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfgnv\" (UniqueName: \"kubernetes.io/projected/df929dfe-655f-468d-a582-cffd181fc0cb-kube-api-access-gfgnv\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:24.934270 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.934152    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-model-cache\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:24.934270 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.934184    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-tmp-dir\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:24.934482 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.934283    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df929dfe-655f-468d-a582-cffd181fc0cb-tls-certs\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:24.934482 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:24.934320    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-home\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.035943 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.035909    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-model-cache\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036112 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.035951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-tmp-dir\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036112 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.035980    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df929dfe-655f-468d-a582-cffd181fc0cb-tls-certs\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036112 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.035996    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-home\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036112 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.036057    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-dshm\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036112 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.036104    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfgnv\" (UniqueName: \"kubernetes.io/projected/df929dfe-655f-468d-a582-cffd181fc0cb-kube-api-access-gfgnv\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036379 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.036304    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-model-cache\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036466 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.036389    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-tmp-dir\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.036550 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.036527    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-home\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.038291 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.038272    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-dshm\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.038569 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.038554    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df929dfe-655f-468d-a582-cffd181fc0cb-tls-certs\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.044037 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.044014    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfgnv\" (UniqueName: \"kubernetes.io/projected/df929dfe-655f-468d-a582-cffd181fc0cb-kube-api-access-gfgnv\") pod \"sub-test-model-0e15bb-kserve-fc547c69-mp722\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") " pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.159926 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.159887    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:25.296791 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:25.296759    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"]
Jul 13 21:41:25.298420 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:41:25.298387    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf929dfe_655f_468d_a582_cffd181fc0cb.slice/crio-452cdda306982501f57effc5b86442cef60b6791bac110a075ebbfae49e56f6a WatchSource:0}: Error finding container 452cdda306982501f57effc5b86442cef60b6791bac110a075ebbfae49e56f6a: Status 404 returned error can't find the container with id 452cdda306982501f57effc5b86442cef60b6791bac110a075ebbfae49e56f6a
Jul 13 21:41:26.271033 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:26.270999    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722" event={"ID":"df929dfe-655f-468d-a582-cffd181fc0cb","Type":"ContainerStarted","Data":"e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234"}
Jul 13 21:41:26.271440 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:26.271043    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722" event={"ID":"df929dfe-655f-468d-a582-cffd181fc0cb","Type":"ContainerStarted","Data":"452cdda306982501f57effc5b86442cef60b6791bac110a075ebbfae49e56f6a"}
Jul 13 21:41:26.288584 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:26.288530    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722" podStartSLOduration=2.28851137 podStartE2EDuration="2.28851137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:26.288068508 +0000 UTC m=+2847.226272838" watchObservedRunningTime="2026-07-13 21:41:26.28851137 +0000 UTC m=+2847.226715681"
Jul 13 21:41:27.776561 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:27.776516    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct" podUID="c6d20229-42a9-465c-a3dc-b050f83b8ba8" containerName="main" containerID="cri-o://29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce" gracePeriod=45
Jul 13 21:41:28.034839 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.034782    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:41:28.164867 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.164831    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-home\") pod \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") "
Jul 13 21:41:28.164867 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.164868    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-model-cache\") pod \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") "
Jul 13 21:41:28.165123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.164887    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-584t2\" (UniqueName: \"kubernetes.io/projected/c6d20229-42a9-465c-a3dc-b050f83b8ba8-kube-api-access-584t2\") pod \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") "
Jul 13 21:41:28.165123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.164921    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tls-certs\") pod \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") "
Jul 13 21:41:28.165123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.164942    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tmp-dir\") pod \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") "
Jul 13 21:41:28.165123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.165044    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-dshm\") pod \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\" (UID: \"c6d20229-42a9-465c-a3dc-b050f83b8ba8\") "
Jul 13 21:41:28.165316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.165171    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6d20229-42a9-465c-a3dc-b050f83b8ba8" (UID: "c6d20229-42a9-465c-a3dc-b050f83b8ba8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:28.165316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.165198    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-home" (OuterVolumeSpecName: "home") pod "c6d20229-42a9-465c-a3dc-b050f83b8ba8" (UID: "c6d20229-42a9-465c-a3dc-b050f83b8ba8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:28.165316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.165239    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6d20229-42a9-465c-a3dc-b050f83b8ba8" (UID: "c6d20229-42a9-465c-a3dc-b050f83b8ba8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:28.165502 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.165379    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:28.165502 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.165392    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:28.165502 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.165402    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:28.167154 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.167130    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-dshm" (OuterVolumeSpecName: "dshm") pod "c6d20229-42a9-465c-a3dc-b050f83b8ba8" (UID: "c6d20229-42a9-465c-a3dc-b050f83b8ba8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:28.167239 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.167167    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d20229-42a9-465c-a3dc-b050f83b8ba8-kube-api-access-584t2" (OuterVolumeSpecName: "kube-api-access-584t2") pod "c6d20229-42a9-465c-a3dc-b050f83b8ba8" (UID: "c6d20229-42a9-465c-a3dc-b050f83b8ba8"). InnerVolumeSpecName "kube-api-access-584t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:41:28.167279 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.167241    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6d20229-42a9-465c-a3dc-b050f83b8ba8" (UID: "c6d20229-42a9-465c-a3dc-b050f83b8ba8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:41:28.248436 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.248401    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:41:28.266755 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.266729    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d20229-42a9-465c-a3dc-b050f83b8ba8-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:28.266755 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.266754    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-584t2\" (UniqueName: \"kubernetes.io/projected/c6d20229-42a9-465c-a3dc-b050f83b8ba8-kube-api-access-584t2\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:28.266755 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.266764    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d20229-42a9-465c-a3dc-b050f83b8ba8-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:28.282832 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.282803    2556 generic.go:356] "Generic (PLEG): container finished" podID="c6d20229-42a9-465c-a3dc-b050f83b8ba8" containerID="29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce" exitCode=0
Jul 13 21:41:28.282992 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.282864    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"
Jul 13 21:41:28.282992 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.282886    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct" event={"ID":"c6d20229-42a9-465c-a3dc-b050f83b8ba8","Type":"ContainerDied","Data":"29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce"}
Jul 13 21:41:28.282992 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.282932    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct" event={"ID":"c6d20229-42a9-465c-a3dc-b050f83b8ba8","Type":"ContainerDied","Data":"0f74ed7e5bbcc0f22f79adf23b0e7212a1faa72c48b8fc43e710ca0630e8dfe1"}
Jul 13 21:41:28.282992 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.282953    2556 scope.go:117] "RemoveContainer" containerID="29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce"
Jul 13 21:41:28.294779 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.294756    2556 scope.go:117] "RemoveContainer" containerID="29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce"
Jul 13 21:41:28.295250 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:41:28.295213    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce\": container with ID starting with 29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce not found: ID does not exist" containerID="29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce"
Jul 13 21:41:28.295387 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.295247    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce"} err="failed to get container status \"29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce\": rpc error: code = NotFound desc = could not find container \"29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce\": container with ID starting with 29b0c99907c9630d0e44f0cd8034bc0344ba1def95b56b0e845825521da667ce not found: ID does not exist"
Jul 13 21:41:28.307775 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.307749    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"]
Jul 13 21:41:28.309947 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.309926    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/auth-test-model-0e15bb-kserve-749947979-cblct"]
Jul 13 21:41:28.843156 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:28.843111    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp" podUID="5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" containerName="main" containerID="cri-o://34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348" gracePeriod=45
Jul 13 21:41:29.100006 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.099941    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:41:29.275747 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.275716    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-dshm\") pod \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") "
Jul 13 21:41:29.275959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.275780    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tmp-dir\") pod \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") "
Jul 13 21:41:29.275959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.275829    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztg6j\" (UniqueName: \"kubernetes.io/projected/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-kube-api-access-ztg6j\") pod \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") "
Jul 13 21:41:29.275959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.275847    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-model-cache\") pod \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") "
Jul 13 21:41:29.275959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.275895    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-home\") pod \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") "
Jul 13 21:41:29.275959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.275945    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tls-certs\") pod \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\" (UID: \"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0\") "
Jul 13 21:41:29.276240 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.276045    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" (UID: "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:29.276240 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.276160    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" (UID: "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:29.276240 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.276189    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-home" (OuterVolumeSpecName: "home") pod "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" (UID: "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:29.276407 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.276300    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:29.276407 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.276315    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:29.276407 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.276329    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:29.278080 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.278054    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-dshm" (OuterVolumeSpecName: "dshm") pod "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" (UID: "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:41:29.278183 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.278102    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-kube-api-access-ztg6j" (OuterVolumeSpecName: "kube-api-access-ztg6j") pod "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" (UID: "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0"). InnerVolumeSpecName "kube-api-access-ztg6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:41:29.278183 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.278124    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" (UID: "5a8eb34c-928b-4860-adb5-bfdc5e78b2f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:41:29.290931 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.290902    2556 generic.go:356] "Generic (PLEG): container finished" podID="5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" containerID="34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348" exitCode=0
Jul 13 21:41:29.291075 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.290949    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"
Jul 13 21:41:29.291075 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.290973    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp" event={"ID":"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0","Type":"ContainerDied","Data":"34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348"}
Jul 13 21:41:29.291075 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.291003    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp" event={"ID":"5a8eb34c-928b-4860-adb5-bfdc5e78b2f0","Type":"ContainerDied","Data":"2ddb9f926de992a9033be0a8f78ab278652f24dd633939bd17af639057bccc14"}
Jul 13 21:41:29.291075 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.291019    2556 scope.go:117] "RemoveContainer" containerID="34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348"
Jul 13 21:41:29.300416 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.300399    2556 scope.go:117] "RemoveContainer" containerID="34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348"
Jul 13 21:41:29.300681 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:41:29.300662    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348\": container with ID starting with 34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348 not found: ID does not exist" containerID="34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348"
Jul 13 21:41:29.300741 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.300696    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348"} err="failed to get container status \"34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348\": rpc error: code = NotFound desc = could not find container \"34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348\": container with ID starting with 34ddd357c253033260b3d204cc2818efa047151cf60af4340b11519ed5770348 not found: ID does not exist"
Jul 13 21:41:29.311626 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.311599    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"]
Jul 13 21:41:29.315628 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.315605    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/auth-test-model-18e186-kserve-65cdd9dfff-jhpwp"]
Jul 13 21:41:29.377478 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.377446    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:29.377478 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.377472    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:29.377478 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.377481    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztg6j\" (UniqueName: \"kubernetes.io/projected/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0-kube-api-access-ztg6j\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:41:29.603346 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.603304    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" path="/var/lib/kubelet/pods/5a8eb34c-928b-4860-adb5-bfdc5e78b2f0/volumes"
Jul 13 21:41:29.603785 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:29.603769    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d20229-42a9-465c-a3dc-b050f83b8ba8" path="/var/lib/kubelet/pods/c6d20229-42a9-465c-a3dc-b050f83b8ba8/volumes"
Jul 13 21:41:35.160427 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:35.160386    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:35.160427 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:35.160434    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:35.173205 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:35.173179    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:35.340230 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:35.340201    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:41:46.955891 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.955855    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"]
Jul 13 21:41:46.956277 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.956269    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" containerName="main"
Jul 13 21:41:46.956316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.956280    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" containerName="main"
Jul 13 21:41:46.956316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.956295    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d20229-42a9-465c-a3dc-b050f83b8ba8" containerName="main"
Jul 13 21:41:46.956316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.956302    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d20229-42a9-465c-a3dc-b050f83b8ba8" containerName="main"
Jul 13 21:41:46.956433 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.956377    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d20229-42a9-465c-a3dc-b050f83b8ba8" containerName="main"
Jul 13 21:41:46.956433 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.956389    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8eb34c-928b-4860-adb5-bfdc5e78b2f0" containerName="main"
Jul 13 21:41:46.959757 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.959736    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:46.963749 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.963727    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"kube-root-ca.crt\""
Jul 13 21:41:46.963749 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.963730    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"sub-test-model-18e186-kserve-self-signed-certs\""
Jul 13 21:41:46.963934 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.963771    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"openshift-service-ca.crt\""
Jul 13 21:41:46.963934 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.963731    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"default-dockercfg-chk48\""
Jul 13 21:41:46.969794 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:46.969770    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"]
Jul 13 21:41:47.034794 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.034759    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-home\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.034794 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.034801    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cscl5\" (UniqueName: \"kubernetes.io/projected/41c67d2e-32ef-43af-a766-55834267ea9c-kube-api-access-cscl5\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.035067 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.034983    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-dshm\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.035067 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.035020    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-model-cache\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.035067 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.035048    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-tmp-dir\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.035218 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.035114    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41c67d2e-32ef-43af-a766-55834267ea9c-tls-certs\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.135752 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.135715    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cscl5\" (UniqueName: \"kubernetes.io/projected/41c67d2e-32ef-43af-a766-55834267ea9c-kube-api-access-cscl5\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.135926 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.135829    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-dshm\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.135926 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.135852    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-model-cache\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.135926 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.135869    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-tmp-dir\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.135926 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.135908    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41c67d2e-32ef-43af-a766-55834267ea9c-tls-certs\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.136166 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.135935    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-home\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.136404 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.136374    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-model-cache\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.136566 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.136404    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-tmp-dir\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.136566 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.136440    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-home\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.138184 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.138164    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-dshm\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.138495 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.138480    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41c67d2e-32ef-43af-a766-55834267ea9c-tls-certs\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.145870 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.145845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cscl5\" (UniqueName: \"kubernetes.io/projected/41c67d2e-32ef-43af-a766-55834267ea9c-kube-api-access-cscl5\") pod \"sub-test-model-18e186-kserve-5cc7449fb5-rsstv\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") " pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.270393 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.270273    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:47.412130 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:47.412101    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"]
Jul 13 21:41:47.414409 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:41:47.414379    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41c67d2e_32ef_43af_a766_55834267ea9c.slice/crio-c378572acda167b665a7ee57769013d79022f696f6e81cf9989db850673ed2ca WatchSource:0}: Error finding container c378572acda167b665a7ee57769013d79022f696f6e81cf9989db850673ed2ca: Status 404 returned error can't find the container with id c378572acda167b665a7ee57769013d79022f696f6e81cf9989db850673ed2ca
Jul 13 21:41:48.403071 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:48.403037    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv" event={"ID":"41c67d2e-32ef-43af-a766-55834267ea9c","Type":"ContainerStarted","Data":"8ae0b2382e84ad167e2f94eab0caadeffc8343b4bda09c0d4b72ad24594a1bcb"}
Jul 13 21:41:48.403501 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:48.403079    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv" event={"ID":"41c67d2e-32ef-43af-a766-55834267ea9c","Type":"ContainerStarted","Data":"c378572acda167b665a7ee57769013d79022f696f6e81cf9989db850673ed2ca"}
Jul 13 21:41:48.422025 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:48.421970    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv" podStartSLOduration=2.421955217 podStartE2EDuration="2.421955217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:48.419222066 +0000 UTC m=+2869.357426372" watchObservedRunningTime="2026-07-13 21:41:48.421955217 +0000 UTC m=+2869.360159529"
Jul 13 21:41:51.859985 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:51.859902    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:41:57.270703 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:57.270663    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:57.271327 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:57.270723    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:57.283081 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:57.283054    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:41:57.465373 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:41:57.465331    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:42:06.545010 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:06.544975    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:42:24.630671 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:24.630628    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"]
Jul 13 21:42:25.464468 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:25.464438    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"]
Jul 13 21:42:28.344144 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:28.344107    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:42:33.849764 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.849728    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"]
Jul 13 21:42:33.853036 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.853020    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:33.855427 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.855408    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e15bb\"/\"rate-test-model-0e15bb-kserve-self-signed-certs\""
Jul 13 21:42:33.863712 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.863687    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"]
Jul 13 21:42:33.959000 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.958961    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkt5b\" (UniqueName: \"kubernetes.io/projected/e83d2b23-578e-40fd-af84-225dff699122-kube-api-access-xkt5b\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:33.959000 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.958999    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-home\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:33.959216 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.959029    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d2b23-578e-40fd-af84-225dff699122-tls-certs\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:33.959216 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.959097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-dshm\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:33.959216 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.959133    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-tmp-dir\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:33.959216 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:33.959174    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-model-cache\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.059780 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.059745    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkt5b\" (UniqueName: \"kubernetes.io/projected/e83d2b23-578e-40fd-af84-225dff699122-kube-api-access-xkt5b\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.059780 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.059783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-home\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.059989 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.059823    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d2b23-578e-40fd-af84-225dff699122-tls-certs\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.059989 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.059848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-dshm\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.059989 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.059875    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-tmp-dir\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.059989 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.059939    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-model-cache\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.060360 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.060315    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-home\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.060438 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.060391    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-model-cache\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.060438 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.060396    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-tmp-dir\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.062182 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.062164    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-dshm\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.062391 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.062374    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d2b23-578e-40fd-af84-225dff699122-tls-certs\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.067651 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.067631    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkt5b\" (UniqueName: \"kubernetes.io/projected/e83d2b23-578e-40fd-af84-225dff699122-kube-api-access-xkt5b\") pod \"rate-test-model-0e15bb-kserve-5bc445659f-wl2pb\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") " pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.162007 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.161972    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:34.294365 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.294320    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"]
Jul 13 21:42:34.295479 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:42:34.295454    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode83d2b23_578e_40fd_af84_225dff699122.slice/crio-2f5cf1cf6b42328d082c210a0b2c5c36fe1037a28337886a2a7e21fc0aac9e6f WatchSource:0}: Error finding container 2f5cf1cf6b42328d082c210a0b2c5c36fe1037a28337886a2a7e21fc0aac9e6f: Status 404 returned error can't find the container with id 2f5cf1cf6b42328d082c210a0b2c5c36fe1037a28337886a2a7e21fc0aac9e6f
Jul 13 21:42:34.667998 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.667966    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb" event={"ID":"e83d2b23-578e-40fd-af84-225dff699122","Type":"ContainerStarted","Data":"531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e"}
Jul 13 21:42:34.668151 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.668012    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb" event={"ID":"e83d2b23-578e-40fd-af84-225dff699122","Type":"ContainerStarted","Data":"2f5cf1cf6b42328d082c210a0b2c5c36fe1037a28337886a2a7e21fc0aac9e6f"}
Jul 13 21:42:34.686940 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:34.686888    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb" podStartSLOduration=1.686873125 podStartE2EDuration="1.686873125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:34.684310856 +0000 UTC m=+2915.622515168" watchObservedRunningTime="2026-07-13 21:42:34.686873125 +0000 UTC m=+2915.625077435"
Jul 13 21:42:39.651052 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:39.651010    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722" podUID="df929dfe-655f-468d-a582-cffd181fc0cb" containerName="main" containerID="cri-o://e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234" gracePeriod=45
Jul 13 21:42:39.901031 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:39.901002    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:42:40.018237 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018147    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df929dfe-655f-468d-a582-cffd181fc0cb-tls-certs\") pod \"df929dfe-655f-468d-a582-cffd181fc0cb\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") "
Jul 13 21:42:40.018237 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018203    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-dshm\") pod \"df929dfe-655f-468d-a582-cffd181fc0cb\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") "
Jul 13 21:42:40.018237 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018232    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfgnv\" (UniqueName: \"kubernetes.io/projected/df929dfe-655f-468d-a582-cffd181fc0cb-kube-api-access-gfgnv\") pod \"df929dfe-655f-468d-a582-cffd181fc0cb\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") "
Jul 13 21:42:40.018554 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018269    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-model-cache\") pod \"df929dfe-655f-468d-a582-cffd181fc0cb\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") "
Jul 13 21:42:40.018554 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018310    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-home\") pod \"df929dfe-655f-468d-a582-cffd181fc0cb\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") "
Jul 13 21:42:40.018554 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018424    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-tmp-dir\") pod \"df929dfe-655f-468d-a582-cffd181fc0cb\" (UID: \"df929dfe-655f-468d-a582-cffd181fc0cb\") "
Jul 13 21:42:40.018725 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018598    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "df929dfe-655f-468d-a582-cffd181fc0cb" (UID: "df929dfe-655f-468d-a582-cffd181fc0cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.018725 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018611    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-home" (OuterVolumeSpecName: "home") pod "df929dfe-655f-468d-a582-cffd181fc0cb" (UID: "df929dfe-655f-468d-a582-cffd181fc0cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.018725 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018683    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df929dfe-655f-468d-a582-cffd181fc0cb" (UID: "df929dfe-655f-468d-a582-cffd181fc0cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.018876 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018828    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.018876 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018845    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.018876 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.018854    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.020405 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.020383    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df929dfe-655f-468d-a582-cffd181fc0cb-kube-api-access-gfgnv" (OuterVolumeSpecName: "kube-api-access-gfgnv") pod "df929dfe-655f-468d-a582-cffd181fc0cb" (UID: "df929dfe-655f-468d-a582-cffd181fc0cb"). InnerVolumeSpecName "kube-api-access-gfgnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:42:40.020405 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.020401    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-dshm" (OuterVolumeSpecName: "dshm") pod "df929dfe-655f-468d-a582-cffd181fc0cb" (UID: "df929dfe-655f-468d-a582-cffd181fc0cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.020548 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.020429    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df929dfe-655f-468d-a582-cffd181fc0cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df929dfe-655f-468d-a582-cffd181fc0cb" (UID: "df929dfe-655f-468d-a582-cffd181fc0cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:42:40.119913 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.119875    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df929dfe-655f-468d-a582-cffd181fc0cb-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.119913 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.119905    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df929dfe-655f-468d-a582-cffd181fc0cb-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.119913 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.119915    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfgnv\" (UniqueName: \"kubernetes.io/projected/df929dfe-655f-468d-a582-cffd181fc0cb-kube-api-access-gfgnv\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.484480 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.484425    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv" podUID="41c67d2e-32ef-43af-a766-55834267ea9c" containerName="main" containerID="cri-o://8ae0b2382e84ad167e2f94eab0caadeffc8343b4bda09c0d4b72ad24594a1bcb" gracePeriod=45
Jul 13 21:42:40.709030 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.708996    2556 generic.go:356] "Generic (PLEG): container finished" podID="41c67d2e-32ef-43af-a766-55834267ea9c" containerID="8ae0b2382e84ad167e2f94eab0caadeffc8343b4bda09c0d4b72ad24594a1bcb" exitCode=0
Jul 13 21:42:40.709460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.709070    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv" event={"ID":"41c67d2e-32ef-43af-a766-55834267ea9c","Type":"ContainerDied","Data":"8ae0b2382e84ad167e2f94eab0caadeffc8343b4bda09c0d4b72ad24594a1bcb"}
Jul 13 21:42:40.710658 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.710630    2556 generic.go:356] "Generic (PLEG): container finished" podID="df929dfe-655f-468d-a582-cffd181fc0cb" containerID="e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234" exitCode=0
Jul 13 21:42:40.710799 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.710673    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"
Jul 13 21:42:40.710799 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.710676    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722" event={"ID":"df929dfe-655f-468d-a582-cffd181fc0cb","Type":"ContainerDied","Data":"e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234"}
Jul 13 21:42:40.710799 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.710706    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722" event={"ID":"df929dfe-655f-468d-a582-cffd181fc0cb","Type":"ContainerDied","Data":"452cdda306982501f57effc5b86442cef60b6791bac110a075ebbfae49e56f6a"}
Jul 13 21:42:40.710799 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.710722    2556 scope.go:117] "RemoveContainer" containerID="e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234"
Jul 13 21:42:40.721912 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.721894    2556 scope.go:117] "RemoveContainer" containerID="e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234"
Jul 13 21:42:40.722199 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:42:40.722182    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234\": container with ID starting with e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234 not found: ID does not exist" containerID="e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234"
Jul 13 21:42:40.722279 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.722207    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234"} err="failed to get container status \"e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234\": rpc error: code = NotFound desc = could not find container \"e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234\": container with ID starting with e4255585fe78fd8dad0c058095e2207ba2ac1068b8e2b93a110ffd993308c234 not found: ID does not exist"
Jul 13 21:42:40.734202 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.734178    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"]
Jul 13 21:42:40.739648 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.739588    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/sub-test-model-0e15bb-kserve-fc547c69-mp722"]
Jul 13 21:42:40.746804 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.746785    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:42:40.826009 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.825976    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-model-cache\") pod \"41c67d2e-32ef-43af-a766-55834267ea9c\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") "
Jul 13 21:42:40.826197 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826018    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-home\") pod \"41c67d2e-32ef-43af-a766-55834267ea9c\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") "
Jul 13 21:42:40.826197 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826047    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41c67d2e-32ef-43af-a766-55834267ea9c-tls-certs\") pod \"41c67d2e-32ef-43af-a766-55834267ea9c\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") "
Jul 13 21:42:40.826197 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826092    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cscl5\" (UniqueName: \"kubernetes.io/projected/41c67d2e-32ef-43af-a766-55834267ea9c-kube-api-access-cscl5\") pod \"41c67d2e-32ef-43af-a766-55834267ea9c\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") "
Jul 13 21:42:40.826197 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826159    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-dshm\") pod \"41c67d2e-32ef-43af-a766-55834267ea9c\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") "
Jul 13 21:42:40.826197 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826193    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-tmp-dir\") pod \"41c67d2e-32ef-43af-a766-55834267ea9c\" (UID: \"41c67d2e-32ef-43af-a766-55834267ea9c\") "
Jul 13 21:42:40.826503 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826273    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-home" (OuterVolumeSpecName: "home") pod "41c67d2e-32ef-43af-a766-55834267ea9c" (UID: "41c67d2e-32ef-43af-a766-55834267ea9c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.826503 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826283    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-model-cache" (OuterVolumeSpecName: "model-cache") pod "41c67d2e-32ef-43af-a766-55834267ea9c" (UID: "41c67d2e-32ef-43af-a766-55834267ea9c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.826596 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826512    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.826596 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826514    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41c67d2e-32ef-43af-a766-55834267ea9c" (UID: "41c67d2e-32ef-43af-a766-55834267ea9c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.826596 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.826527    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.828396 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.828373    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-dshm" (OuterVolumeSpecName: "dshm") pod "41c67d2e-32ef-43af-a766-55834267ea9c" (UID: "41c67d2e-32ef-43af-a766-55834267ea9c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:40.828396 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.828382    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41c67d2e-32ef-43af-a766-55834267ea9c-kube-api-access-cscl5" (OuterVolumeSpecName: "kube-api-access-cscl5") pod "41c67d2e-32ef-43af-a766-55834267ea9c" (UID: "41c67d2e-32ef-43af-a766-55834267ea9c"). InnerVolumeSpecName "kube-api-access-cscl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:42:40.828568 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.828408    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c67d2e-32ef-43af-a766-55834267ea9c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41c67d2e-32ef-43af-a766-55834267ea9c" (UID: "41c67d2e-32ef-43af-a766-55834267ea9c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:42:40.927159 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.927116    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41c67d2e-32ef-43af-a766-55834267ea9c-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.927159 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.927158    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cscl5\" (UniqueName: \"kubernetes.io/projected/41c67d2e-32ef-43af-a766-55834267ea9c-kube-api-access-cscl5\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.927409 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.927173    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:40.927409 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:40.927185    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c67d2e-32ef-43af-a766-55834267ea9c-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:42:41.051879 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:41.051785    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:42:41.604932 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:41.604898    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df929dfe-655f-468d-a582-cffd181fc0cb" path="/var/lib/kubelet/pods/df929dfe-655f-468d-a582-cffd181fc0cb/volumes"
Jul 13 21:42:41.718031 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:41.717999    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv" event={"ID":"41c67d2e-32ef-43af-a766-55834267ea9c","Type":"ContainerDied","Data":"c378572acda167b665a7ee57769013d79022f696f6e81cf9989db850673ed2ca"}
Jul 13 21:42:41.718031 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:41.718021    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"
Jul 13 21:42:41.718571 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:41.718044    2556 scope.go:117] "RemoveContainer" containerID="8ae0b2382e84ad167e2f94eab0caadeffc8343b4bda09c0d4b72ad24594a1bcb"
Jul 13 21:42:41.737556 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:41.737529    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"]
Jul 13 21:42:41.744104 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:41.744075    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/sub-test-model-18e186-kserve-5cc7449fb5-rsstv"]
Jul 13 21:42:43.603585 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:43.603550    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41c67d2e-32ef-43af-a766-55834267ea9c" path="/var/lib/kubelet/pods/41c67d2e-32ef-43af-a766-55834267ea9c/volumes"
Jul 13 21:42:44.162311 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:44.162270    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:44.162311 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:44.162311    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:44.174991 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:44.174966    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:44.747647 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:44.747622    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:42:55.030068 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.030035    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"]
Jul 13 21:42:55.030627 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.030608    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df929dfe-655f-468d-a582-cffd181fc0cb" containerName="main"
Jul 13 21:42:55.030711 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.030629    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="df929dfe-655f-468d-a582-cffd181fc0cb" containerName="main"
Jul 13 21:42:55.030711 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.030645    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41c67d2e-32ef-43af-a766-55834267ea9c" containerName="main"
Jul 13 21:42:55.030711 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.030653    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c67d2e-32ef-43af-a766-55834267ea9c" containerName="main"
Jul 13 21:42:55.030863 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.030759    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="df929dfe-655f-468d-a582-cffd181fc0cb" containerName="main"
Jul 13 21:42:55.030863 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.030772    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="41c67d2e-32ef-43af-a766-55834267ea9c" containerName="main"
Jul 13 21:42:55.035168 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.035148    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.039031 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.039014    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"openshift-service-ca.crt\""
Jul 13 21:42:55.039288 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.039272    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"default-dockercfg-chk48\""
Jul 13 21:42:55.039288 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.039279    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"kube-root-ca.crt\""
Jul 13 21:42:55.039456 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.039295    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-18e186\"/\"rate-test-model-18e186-kserve-self-signed-certs\""
Jul 13 21:42:55.043996 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.043972    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"]
Jul 13 21:42:55.166011 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.165978    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-home\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.166194 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.166018    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tmp-dir\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.166194 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.166097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-model-cache\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.166194 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.166146    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tls-certs\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.166315 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.166202    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-dshm\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.166315 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.166225    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxsfn\" (UniqueName: \"kubernetes.io/projected/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-kube-api-access-kxsfn\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267151 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267118    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-home\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267151 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267157    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tmp-dir\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267424 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-model-cache\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267424 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267224    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tls-certs\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267424 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267276    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-dshm\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267424 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267308    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxsfn\" (UniqueName: \"kubernetes.io/projected/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-kube-api-access-kxsfn\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267635 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267555    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-home\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267635 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267589    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-model-cache\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.267635 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.267628    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tmp-dir\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.269513 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.269490    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-dshm\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.269765 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.269748    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tls-certs\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.276043 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.276017    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxsfn\" (UniqueName: \"kubernetes.io/projected/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-kube-api-access-kxsfn\") pod \"rate-test-model-18e186-kserve-b74dfc7ff-cs7mx\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") " pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.345118 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.345018    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:42:55.482675 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.482648    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"]
Jul 13 21:42:55.484268 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:42:55.484240    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d0e5840_de72_4c2a_b3d2_1f639f3155a5.slice/crio-f2388e1e165098ddab0503e134267b4c84d1a7c454e99e3ad5aecf04f9a677e3 WatchSource:0}: Error finding container f2388e1e165098ddab0503e134267b4c84d1a7c454e99e3ad5aecf04f9a677e3: Status 404 returned error can't find the container with id f2388e1e165098ddab0503e134267b4c84d1a7c454e99e3ad5aecf04f9a677e3
Jul 13 21:42:55.798370 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.798318    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx" event={"ID":"3d0e5840-de72-4c2a-b3d2-1f639f3155a5","Type":"ContainerStarted","Data":"4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239"}
Jul 13 21:42:55.798522 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.798385    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx" event={"ID":"3d0e5840-de72-4c2a-b3d2-1f639f3155a5","Type":"ContainerStarted","Data":"f2388e1e165098ddab0503e134267b4c84d1a7c454e99e3ad5aecf04f9a677e3"}
Jul 13 21:42:55.815203 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:42:55.815158    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx" podStartSLOduration=0.815146455 podStartE2EDuration="815.146455ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:55.814295632 +0000 UTC m=+2936.752499943" watchObservedRunningTime="2026-07-13 21:42:55.815146455 +0000 UTC m=+2936.753350766"
Jul 13 21:43:00.344832 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:00.344793    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:43:05.345592 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:05.345556    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:43:05.346106 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:05.345609    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:43:05.358006 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:05.357977    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:43:05.871597 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:05.871569    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:43:15.247940 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:15.247890    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:43:35.341457 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:35.341419    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:43:41.543732 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:41.543699    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"]
Jul 13 21:43:42.426316 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:42.426284    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"]
Jul 13 21:43:43.416689 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.416655    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7575-140194763941936"]
Jul 13 21:43:43.420098 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.420081    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:43.422522 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.422500    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-h48p7\""
Jul 13 21:43:43.426707 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.426686    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941936"]
Jul 13 21:43:43.432967 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.432941    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkrts\" (UniqueName: \"kubernetes.io/projected/2718ca42-1539-41e2-b3d3-e368284d403b-kube-api-access-xkrts\") pod \"test-curl-7575-140194763941936\" (UID: \"2718ca42-1539-41e2-b3d3-e368284d403b\") " pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:43.534227 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.534186    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkrts\" (UniqueName: \"kubernetes.io/projected/2718ca42-1539-41e2-b3d3-e368284d403b-kube-api-access-xkrts\") pod \"test-curl-7575-140194763941936\" (UID: \"2718ca42-1539-41e2-b3d3-e368284d403b\") " pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:43.542498 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.542466    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkrts\" (UniqueName: \"kubernetes.io/projected/2718ca42-1539-41e2-b3d3-e368284d403b-kube-api-access-xkrts\") pod \"test-curl-7575-140194763941936\" (UID: \"2718ca42-1539-41e2-b3d3-e368284d403b\") " pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:43.731079 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.730988    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:43.863790 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.863762    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941936"]
Jul 13 21:43:43.865396 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:43:43.865369    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2718ca42_1539_41e2_b3d3_e368284d403b.slice/crio-d3c805988a1bacb30b8ccb4312ca58e61ef1d00b20cd1d518e4d56bba51bd365 WatchSource:0}: Error finding container d3c805988a1bacb30b8ccb4312ca58e61ef1d00b20cd1d518e4d56bba51bd365: Status 404 returned error can't find the container with id d3c805988a1bacb30b8ccb4312ca58e61ef1d00b20cd1d518e4d56bba51bd365
Jul 13 21:43:43.867122 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:43.867103    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:43:44.086877 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:44.086788    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763941936" event={"ID":"2718ca42-1539-41e2-b3d3-e368284d403b","Type":"ContainerStarted","Data":"d3c805988a1bacb30b8ccb4312ca58e61ef1d00b20cd1d518e4d56bba51bd365"}
Jul 13 21:43:45.094042 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:45.094003    2556 generic.go:356] "Generic (PLEG): container finished" podID="2718ca42-1539-41e2-b3d3-e368284d403b" containerID="f58515ec75869f3693079290778f9d710c80a43baf158fdf4e48079d07bb1d2a" exitCode=0
Jul 13 21:43:45.094450 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:45.094089    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763941936" event={"ID":"2718ca42-1539-41e2-b3d3-e368284d403b","Type":"ContainerDied","Data":"f58515ec75869f3693079290778f9d710c80a43baf158fdf4e48079d07bb1d2a"}
Jul 13 21:43:46.239561 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.239534    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:46.259184 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.259155    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkrts\" (UniqueName: \"kubernetes.io/projected/2718ca42-1539-41e2-b3d3-e368284d403b-kube-api-access-xkrts\") pod \"2718ca42-1539-41e2-b3d3-e368284d403b\" (UID: \"2718ca42-1539-41e2-b3d3-e368284d403b\") "
Jul 13 21:43:46.261314 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.261282    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2718ca42-1539-41e2-b3d3-e368284d403b-kube-api-access-xkrts" (OuterVolumeSpecName: "kube-api-access-xkrts") pod "2718ca42-1539-41e2-b3d3-e368284d403b" (UID: "2718ca42-1539-41e2-b3d3-e368284d403b"). InnerVolumeSpecName "kube-api-access-xkrts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:46.279317 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.273402    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7575-140194763941936_2718ca42-1539-41e2-b3d3-e368284d403b/test-curl-7575-140194763941936/0.log"
Jul 13 21:43:46.296881 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.296851    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941936"]
Jul 13 21:43:46.299853 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.299831    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941936"]
Jul 13 21:43:46.360371 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.360306    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkrts\" (UniqueName: \"kubernetes.io/projected/2718ca42-1539-41e2-b3d3-e368284d403b-kube-api-access-xkrts\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:46.423302 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.423264    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7575-140194763941552"]
Jul 13 21:43:46.423763 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.423746    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2718ca42-1539-41e2-b3d3-e368284d403b" containerName="test-curl-7575-140194763941936"
Jul 13 21:43:46.423763 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.423763    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2718ca42-1539-41e2-b3d3-e368284d403b" containerName="test-curl-7575-140194763941936"
Jul 13 21:43:46.423893 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.423831    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2718ca42-1539-41e2-b3d3-e368284d403b" containerName="test-curl-7575-140194763941936"
Jul 13 21:43:46.426941 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.426925    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941552"
Jul 13 21:43:46.430655 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.430627    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941552"]
Jul 13 21:43:46.461353 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.461314    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j98r\" (UniqueName: \"kubernetes.io/projected/a4c0bb0f-13b2-4396-b46a-39dacdd11eba-kube-api-access-9j98r\") pod \"test-curl-7575-140194763941552\" (UID: \"a4c0bb0f-13b2-4396-b46a-39dacdd11eba\") " pod="openshift-ingress/test-curl-7575-140194763941552"
Jul 13 21:43:46.562196 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.562162    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j98r\" (UniqueName: \"kubernetes.io/projected/a4c0bb0f-13b2-4396-b46a-39dacdd11eba-kube-api-access-9j98r\") pod \"test-curl-7575-140194763941552\" (UID: \"a4c0bb0f-13b2-4396-b46a-39dacdd11eba\") " pod="openshift-ingress/test-curl-7575-140194763941552"
Jul 13 21:43:46.570161 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.570129    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j98r\" (UniqueName: \"kubernetes.io/projected/a4c0bb0f-13b2-4396-b46a-39dacdd11eba-kube-api-access-9j98r\") pod \"test-curl-7575-140194763941552\" (UID: \"a4c0bb0f-13b2-4396-b46a-39dacdd11eba\") " pod="openshift-ingress/test-curl-7575-140194763941552"
Jul 13 21:43:46.737612 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.737515    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941552"
Jul 13 21:43:46.877440 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:46.877417    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941552"]
Jul 13 21:43:46.879583 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:43:46.879554    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4c0bb0f_13b2_4396_b46a_39dacdd11eba.slice/crio-4c5939ce90e08011903d6128458c1b2b2b67133aacaae20cbf92196551b7f1f5 WatchSource:0}: Error finding container 4c5939ce90e08011903d6128458c1b2b2b67133aacaae20cbf92196551b7f1f5: Status 404 returned error can't find the container with id 4c5939ce90e08011903d6128458c1b2b2b67133aacaae20cbf92196551b7f1f5
Jul 13 21:43:47.106156 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:47.106070    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:47.106156 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:47.106094    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3c805988a1bacb30b8ccb4312ca58e61ef1d00b20cd1d518e4d56bba51bd365"
Jul 13 21:43:47.107298 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:47.107272    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763941552" event={"ID":"a4c0bb0f-13b2-4396-b46a-39dacdd11eba","Type":"ContainerStarted","Data":"4c5939ce90e08011903d6128458c1b2b2b67133aacaae20cbf92196551b7f1f5"}
Jul 13 21:43:47.117199 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:43:47.117165    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7575-140194763941936\" is forbidden: User \"system:node:ip-10-0-138-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-146.ec2.internal' and this object" podUID="2718ca42-1539-41e2-b3d3-e368284d403b" pod="openshift-ingress/test-curl-7575-140194763941936"
Jul 13 21:43:47.602831 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:47.602798    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2718ca42-1539-41e2-b3d3-e368284d403b" path="/var/lib/kubelet/pods/2718ca42-1539-41e2-b3d3-e368284d403b/volumes"
Jul 13 21:43:48.114418 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:48.114383    2556 generic.go:356] "Generic (PLEG): container finished" podID="a4c0bb0f-13b2-4396-b46a-39dacdd11eba" containerID="9e77118a29fe8a427ed26df9c3083246bcd42707bcec1d5c1703c90684444740" exitCode=0
Jul 13 21:43:48.114418 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:48.114422    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763941552" event={"ID":"a4c0bb0f-13b2-4396-b46a-39dacdd11eba","Type":"ContainerDied","Data":"9e77118a29fe8a427ed26df9c3083246bcd42707bcec1d5c1703c90684444740"}
Jul 13 21:43:49.261130 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.261105    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941552"
Jul 13 21:43:49.280599 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.280569    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7575-140194763941552_a4c0bb0f-13b2-4396-b46a-39dacdd11eba/test-curl-7575-140194763941552/0.log"
Jul 13 21:43:49.288719 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.288692    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j98r\" (UniqueName: \"kubernetes.io/projected/a4c0bb0f-13b2-4396-b46a-39dacdd11eba-kube-api-access-9j98r\") pod \"a4c0bb0f-13b2-4396-b46a-39dacdd11eba\" (UID: \"a4c0bb0f-13b2-4396-b46a-39dacdd11eba\") "
Jul 13 21:43:49.290695 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.290667    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c0bb0f-13b2-4396-b46a-39dacdd11eba-kube-api-access-9j98r" (OuterVolumeSpecName: "kube-api-access-9j98r") pod "a4c0bb0f-13b2-4396-b46a-39dacdd11eba" (UID: "a4c0bb0f-13b2-4396-b46a-39dacdd11eba"). InnerVolumeSpecName "kube-api-access-9j98r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:49.304708 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.304677    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941552"]
Jul 13 21:43:49.308797 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.308771    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7575-140194763941552"]
Jul 13 21:43:49.389729 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.389644    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j98r\" (UniqueName: \"kubernetes.io/projected/a4c0bb0f-13b2-4396-b46a-39dacdd11eba-kube-api-access-9j98r\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:49.469192 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.469148    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:49.469638 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.469619    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c0bb0f-13b2-4396-b46a-39dacdd11eba" containerName="test-curl-7575-140194763941552"
Jul 13 21:43:49.469749 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.469640    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c0bb0f-13b2-4396-b46a-39dacdd11eba" containerName="test-curl-7575-140194763941552"
Jul 13 21:43:49.469749 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.469737    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4c0bb0f-13b2-4396-b46a-39dacdd11eba" containerName="test-curl-7575-140194763941552"
Jul 13 21:43:49.472784 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.472762    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:49.478311 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.478291    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:49.490984 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.490957    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcnfh\" (UniqueName: \"kubernetes.io/projected/8b269d5d-c7bb-4984-ba5c-18c4311a3532-kube-api-access-rcnfh\") pod \"test-curl-7575-140194763917904\" (UID: \"8b269d5d-c7bb-4984-ba5c-18c4311a3532\") " pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:49.592120 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.592090    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcnfh\" (UniqueName: \"kubernetes.io/projected/8b269d5d-c7bb-4984-ba5c-18c4311a3532-kube-api-access-rcnfh\") pod \"test-curl-7575-140194763917904\" (UID: \"8b269d5d-c7bb-4984-ba5c-18c4311a3532\") " pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:49.606361 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.600400    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcnfh\" (UniqueName: \"kubernetes.io/projected/8b269d5d-c7bb-4984-ba5c-18c4311a3532-kube-api-access-rcnfh\") pod \"test-curl-7575-140194763917904\" (UID: \"8b269d5d-c7bb-4984-ba5c-18c4311a3532\") " pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:49.609533 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.609506    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4c0bb0f-13b2-4396-b46a-39dacdd11eba" path="/var/lib/kubelet/pods/a4c0bb0f-13b2-4396-b46a-39dacdd11eba/volumes"
Jul 13 21:43:49.783749 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.783670    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:49.914055 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:49.914035    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:49.915383 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:43:49.915355    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b269d5d_c7bb_4984_ba5c_18c4311a3532.slice/crio-78790b2db7863191c92d21623abf12e3ae0cca44646ad9c11575b299c40ab720 WatchSource:0}: Error finding container 78790b2db7863191c92d21623abf12e3ae0cca44646ad9c11575b299c40ab720: Status 404 returned error can't find the container with id 78790b2db7863191c92d21623abf12e3ae0cca44646ad9c11575b299c40ab720
Jul 13 21:43:50.127438 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:50.127407    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763941552"
Jul 13 21:43:50.127621 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:50.127438    2556 scope.go:117] "RemoveContainer" containerID="9e77118a29fe8a427ed26df9c3083246bcd42707bcec1d5c1703c90684444740"
Jul 13 21:43:50.128818 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:50.128793    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763917904" event={"ID":"8b269d5d-c7bb-4984-ba5c-18c4311a3532","Type":"ContainerStarted","Data":"78790b2db7863191c92d21623abf12e3ae0cca44646ad9c11575b299c40ab720"}
Jul 13 21:43:51.138062 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:51.138026    2556 generic.go:356] "Generic (PLEG): container finished" podID="8b269d5d-c7bb-4984-ba5c-18c4311a3532" containerID="bdf918cecad3ca6e5a4eb72e7f14595c59b47192e9b9f8c5e5e193618b6fa11d" exitCode=0
Jul 13 21:43:51.138531 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:51.138116    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763917904" event={"ID":"8b269d5d-c7bb-4984-ba5c-18c4311a3532","Type":"ContainerDied","Data":"bdf918cecad3ca6e5a4eb72e7f14595c59b47192e9b9f8c5e5e193618b6fa11d"}
Jul 13 21:43:52.283962 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.283938    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:52.303439 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.303403    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7575-140194763917904_8b269d5d-c7bb-4984-ba5c-18c4311a3532/test-curl-7575-140194763917904/0.log"
Jul 13 21:43:52.317713 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.317684    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcnfh\" (UniqueName: \"kubernetes.io/projected/8b269d5d-c7bb-4984-ba5c-18c4311a3532-kube-api-access-rcnfh\") pod \"8b269d5d-c7bb-4984-ba5c-18c4311a3532\" (UID: \"8b269d5d-c7bb-4984-ba5c-18c4311a3532\") "
Jul 13 21:43:52.319678 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.319653    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b269d5d-c7bb-4984-ba5c-18c4311a3532-kube-api-access-rcnfh" (OuterVolumeSpecName: "kube-api-access-rcnfh") pod "8b269d5d-c7bb-4984-ba5c-18c4311a3532" (UID: "8b269d5d-c7bb-4984-ba5c-18c4311a3532"). InnerVolumeSpecName "kube-api-access-rcnfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:52.327568 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.327544    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:52.330590 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.330559    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:52.419001 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.418901    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcnfh\" (UniqueName: \"kubernetes.io/projected/8b269d5d-c7bb-4984-ba5c-18c4311a3532-kube-api-access-rcnfh\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:52.451789 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.451759    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:52.452165 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.452144    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b269d5d-c7bb-4984-ba5c-18c4311a3532" containerName="test-curl-7575-140194763917904"
Jul 13 21:43:52.452165 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.452158    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b269d5d-c7bb-4984-ba5c-18c4311a3532" containerName="test-curl-7575-140194763917904"
Jul 13 21:43:52.452261 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.452234    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b269d5d-c7bb-4984-ba5c-18c4311a3532" containerName="test-curl-7575-140194763917904"
Jul 13 21:43:52.455269 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.455254    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:52.462639 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.462612    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:52.519409 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.519373    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shtmm\" (UniqueName: \"kubernetes.io/projected/53d5a301-8496-45bc-af21-44872183f6f0-kube-api-access-shtmm\") pod \"test-curl-7575-140194763917904\" (UID: \"53d5a301-8496-45bc-af21-44872183f6f0\") " pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:52.620346 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.620312    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shtmm\" (UniqueName: \"kubernetes.io/projected/53d5a301-8496-45bc-af21-44872183f6f0-kube-api-access-shtmm\") pod \"test-curl-7575-140194763917904\" (UID: \"53d5a301-8496-45bc-af21-44872183f6f0\") " pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:52.628180 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.628154    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shtmm\" (UniqueName: \"kubernetes.io/projected/53d5a301-8496-45bc-af21-44872183f6f0-kube-api-access-shtmm\") pod \"test-curl-7575-140194763917904\" (UID: \"53d5a301-8496-45bc-af21-44872183f6f0\") " pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:52.765872 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.765781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:52.897832 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:52.897804    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:52.899137 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:43:52.899113    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53d5a301_8496_45bc_af21_44872183f6f0.slice/crio-2d425989fe3a305127b11b89f8ca9731d344dee100bcfd51a309a75bc8fe4914 WatchSource:0}: Error finding container 2d425989fe3a305127b11b89f8ca9731d344dee100bcfd51a309a75bc8fe4914: Status 404 returned error can't find the container with id 2d425989fe3a305127b11b89f8ca9731d344dee100bcfd51a309a75bc8fe4914
Jul 13 21:43:53.152216 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:53.152178    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763917904" event={"ID":"53d5a301-8496-45bc-af21-44872183f6f0","Type":"ContainerStarted","Data":"2d425989fe3a305127b11b89f8ca9731d344dee100bcfd51a309a75bc8fe4914"}
Jul 13 21:43:53.153371 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:53.153329    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:53.153502 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:53.153381    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78790b2db7863191c92d21623abf12e3ae0cca44646ad9c11575b299c40ab720"
Jul 13 21:43:53.165424 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:53.165393    2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7575-140194763917904" oldPodUID="8b269d5d-c7bb-4984-ba5c-18c4311a3532" podUID="53d5a301-8496-45bc-af21-44872183f6f0"
Jul 13 21:43:53.607931 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:53.607839    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b269d5d-c7bb-4984-ba5c-18c4311a3532" path="/var/lib/kubelet/pods/8b269d5d-c7bb-4984-ba5c-18c4311a3532/volumes"
Jul 13 21:43:54.160982 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:54.160943    2556 generic.go:356] "Generic (PLEG): container finished" podID="53d5a301-8496-45bc-af21-44872183f6f0" containerID="a96b7881b7689779150e550f5001e34969db19048e24e57076b9ad5f1890ad26" exitCode=0
Jul 13 21:43:54.161167 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:54.161020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763917904" event={"ID":"53d5a301-8496-45bc-af21-44872183f6f0","Type":"ContainerDied","Data":"a96b7881b7689779150e550f5001e34969db19048e24e57076b9ad5f1890ad26"}
Jul 13 21:43:55.336460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.336432    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:55.357506 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.357475    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7575-140194763917904_53d5a301-8496-45bc-af21-44872183f6f0/test-curl-7575-140194763917904/0.log"
Jul 13 21:43:55.379981 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.379947    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:55.385587 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.385563    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7575-140194763917904"]
Jul 13 21:43:55.447331 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.447250    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shtmm\" (UniqueName: \"kubernetes.io/projected/53d5a301-8496-45bc-af21-44872183f6f0-kube-api-access-shtmm\") pod \"53d5a301-8496-45bc-af21-44872183f6f0\" (UID: \"53d5a301-8496-45bc-af21-44872183f6f0\") "
Jul 13 21:43:55.449355 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.449324    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53d5a301-8496-45bc-af21-44872183f6f0-kube-api-access-shtmm" (OuterVolumeSpecName: "kube-api-access-shtmm") pod "53d5a301-8496-45bc-af21-44872183f6f0" (UID: "53d5a301-8496-45bc-af21-44872183f6f0"). InnerVolumeSpecName "kube-api-access-shtmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:55.508032 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.507989    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:55.508399 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.508386    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d5a301-8496-45bc-af21-44872183f6f0" containerName="test-curl-7575-140194763917904"
Jul 13 21:43:55.508451 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.508400    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d5a301-8496-45bc-af21-44872183f6f0" containerName="test-curl-7575-140194763917904"
Jul 13 21:43:55.508487 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.508468    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d5a301-8496-45bc-af21-44872183f6f0" containerName="test-curl-7575-140194763917904"
Jul 13 21:43:55.511405 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.511390    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:55.518731 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.518709    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:55.548035 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.548004    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd9jq\" (UniqueName: \"kubernetes.io/projected/5ed6ea53-b782-4483-96f6-396761d31957-kube-api-access-bd9jq\") pod \"test-curl-7575-140194763918912\" (UID: \"5ed6ea53-b782-4483-96f6-396761d31957\") " pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:55.548205 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.548112    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shtmm\" (UniqueName: \"kubernetes.io/projected/53d5a301-8496-45bc-af21-44872183f6f0-kube-api-access-shtmm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:55.603065 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.603025    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d5a301-8496-45bc-af21-44872183f6f0" path="/var/lib/kubelet/pods/53d5a301-8496-45bc-af21-44872183f6f0/volumes"
Jul 13 21:43:55.649125 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.649096    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd9jq\" (UniqueName: \"kubernetes.io/projected/5ed6ea53-b782-4483-96f6-396761d31957-kube-api-access-bd9jq\") pod \"test-curl-7575-140194763918912\" (UID: \"5ed6ea53-b782-4483-96f6-396761d31957\") " pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:55.658089 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.658060    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd9jq\" (UniqueName: \"kubernetes.io/projected/5ed6ea53-b782-4483-96f6-396761d31957-kube-api-access-bd9jq\") pod \"test-curl-7575-140194763918912\" (UID: \"5ed6ea53-b782-4483-96f6-396761d31957\") " pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:55.821528 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.821434    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:55.944067 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.944036    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:43:55.956006 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:55.955982    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:55.957754 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:43:55.957723    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ed6ea53_b782_4483_96f6_396761d31957.slice/crio-178ffc4a12c3b91bafcdd7f3afe653af2b6e9e8af1f332ef43d3b47a1bf367ba WatchSource:0}: Error finding container 178ffc4a12c3b91bafcdd7f3afe653af2b6e9e8af1f332ef43d3b47a1bf367ba: Status 404 returned error can't find the container with id 178ffc4a12c3b91bafcdd7f3afe653af2b6e9e8af1f332ef43d3b47a1bf367ba
Jul 13 21:43:56.173092 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.173061    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763917904"
Jul 13 21:43:56.173300 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.173118    2556 scope.go:117] "RemoveContainer" containerID="a96b7881b7689779150e550f5001e34969db19048e24e57076b9ad5f1890ad26"
Jul 13 21:43:56.174526 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.174498    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763918912" event={"ID":"5ed6ea53-b782-4483-96f6-396761d31957","Type":"ContainerStarted","Data":"178ffc4a12c3b91bafcdd7f3afe653af2b6e9e8af1f332ef43d3b47a1bf367ba"}
Jul 13 21:43:56.564471 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.564376    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb" podUID="e83d2b23-578e-40fd-af84-225dff699122" containerName="main" containerID="cri-o://531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e" gracePeriod=45
Jul 13 21:43:56.824581 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.824510    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:43:56.862477 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862449    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-model-cache\") pod \"e83d2b23-578e-40fd-af84-225dff699122\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") "
Jul 13 21:43:56.862648 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862492    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-dshm\") pod \"e83d2b23-578e-40fd-af84-225dff699122\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") "
Jul 13 21:43:56.862648 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862534    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d2b23-578e-40fd-af84-225dff699122-tls-certs\") pod \"e83d2b23-578e-40fd-af84-225dff699122\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") "
Jul 13 21:43:56.862648 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862566    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkt5b\" (UniqueName: \"kubernetes.io/projected/e83d2b23-578e-40fd-af84-225dff699122-kube-api-access-xkt5b\") pod \"e83d2b23-578e-40fd-af84-225dff699122\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") "
Jul 13 21:43:56.862648 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862601    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-home\") pod \"e83d2b23-578e-40fd-af84-225dff699122\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") "
Jul 13 21:43:56.862859 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862728    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-tmp-dir\") pod \"e83d2b23-578e-40fd-af84-225dff699122\" (UID: \"e83d2b23-578e-40fd-af84-225dff699122\") "
Jul 13 21:43:56.862859 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862758    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-model-cache" (OuterVolumeSpecName: "model-cache") pod "e83d2b23-578e-40fd-af84-225dff699122" (UID: "e83d2b23-578e-40fd-af84-225dff699122"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:56.862971 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862926    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-home" (OuterVolumeSpecName: "home") pod "e83d2b23-578e-40fd-af84-225dff699122" (UID: "e83d2b23-578e-40fd-af84-225dff699122"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:56.863016 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.862968    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e83d2b23-578e-40fd-af84-225dff699122" (UID: "e83d2b23-578e-40fd-af84-225dff699122"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:56.863016 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.863006    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:56.864716 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.864678    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-dshm" (OuterVolumeSpecName: "dshm") pod "e83d2b23-578e-40fd-af84-225dff699122" (UID: "e83d2b23-578e-40fd-af84-225dff699122"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:56.864814 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.864747    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83d2b23-578e-40fd-af84-225dff699122-kube-api-access-xkt5b" (OuterVolumeSpecName: "kube-api-access-xkt5b") pod "e83d2b23-578e-40fd-af84-225dff699122" (UID: "e83d2b23-578e-40fd-af84-225dff699122"). InnerVolumeSpecName "kube-api-access-xkt5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:56.864910 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.864890    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e83d2b23-578e-40fd-af84-225dff699122-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e83d2b23-578e-40fd-af84-225dff699122" (UID: "e83d2b23-578e-40fd-af84-225dff699122"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:43:56.963769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.963731    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:56.963769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.963762    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:56.963769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.963772    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83d2b23-578e-40fd-af84-225dff699122-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:56.963769 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.963780    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkt5b\" (UniqueName: \"kubernetes.io/projected/e83d2b23-578e-40fd-af84-225dff699122-kube-api-access-xkt5b\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:56.964037 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:56.963789    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83d2b23-578e-40fd-af84-225dff699122-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:57.182227 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.182184    2556 generic.go:356] "Generic (PLEG): container finished" podID="e83d2b23-578e-40fd-af84-225dff699122" containerID="531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e" exitCode=0
Jul 13 21:43:57.182227 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.182228    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"
Jul 13 21:43:57.182518 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.182255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb" event={"ID":"e83d2b23-578e-40fd-af84-225dff699122","Type":"ContainerDied","Data":"531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e"}
Jul 13 21:43:57.182518 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.182293    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb" event={"ID":"e83d2b23-578e-40fd-af84-225dff699122","Type":"ContainerDied","Data":"2f5cf1cf6b42328d082c210a0b2c5c36fe1037a28337886a2a7e21fc0aac9e6f"}
Jul 13 21:43:57.182518 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.182313    2556 scope.go:117] "RemoveContainer" containerID="531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e"
Jul 13 21:43:57.185306 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.185279    2556 generic.go:356] "Generic (PLEG): container finished" podID="5ed6ea53-b782-4483-96f6-396761d31957" containerID="9b1dc442c76b58d291e0efd97acd0bde31e1c181e3a9c64b2d24533f0eb60284" exitCode=0
Jul 13 21:43:57.185451 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.185320    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763918912" event={"ID":"5ed6ea53-b782-4483-96f6-396761d31957","Type":"ContainerDied","Data":"9b1dc442c76b58d291e0efd97acd0bde31e1c181e3a9c64b2d24533f0eb60284"}
Jul 13 21:43:57.192545 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.192529    2556 scope.go:117] "RemoveContainer" containerID="531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e"
Jul 13 21:43:57.192790 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:43:57.192772    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e\": container with ID starting with 531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e not found: ID does not exist" containerID="531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e"
Jul 13 21:43:57.192843 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.192795    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e"} err="failed to get container status \"531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e\": rpc error: code = NotFound desc = could not find container \"531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e\": container with ID starting with 531b15c84b6212b8f3c822db4ba20710496546d0a3de20643c490a67b840d98e not found: ID does not exist"
Jul 13 21:43:57.215730 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.215696    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"]
Jul 13 21:43:57.220894 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.220872    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0e15bb/rate-test-model-0e15bb-kserve-5bc445659f-wl2pb"]
Jul 13 21:43:57.447288 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.447177    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx" podUID="3d0e5840-de72-4c2a-b3d2-1f639f3155a5" containerName="main" containerID="cri-o://4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239" gracePeriod=45
Jul 13 21:43:57.603589 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.603555    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e83d2b23-578e-40fd-af84-225dff699122" path="/var/lib/kubelet/pods/e83d2b23-578e-40fd-af84-225dff699122/volumes"
Jul 13 21:43:57.703545 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.703489    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:43:57.770434 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770405    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxsfn\" (UniqueName: \"kubernetes.io/projected/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-kube-api-access-kxsfn\") pod \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") "
Jul 13 21:43:57.770615 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770455    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-dshm\") pod \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") "
Jul 13 21:43:57.770615 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770479    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-model-cache\") pod \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") "
Jul 13 21:43:57.770615 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770497    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tls-certs\") pod \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") "
Jul 13 21:43:57.770615 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770525    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tmp-dir\") pod \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") "
Jul 13 21:43:57.770615 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770554    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-home\") pod \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\" (UID: \"3d0e5840-de72-4c2a-b3d2-1f639f3155a5\") "
Jul 13 21:43:57.770878 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770801    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d0e5840-de72-4c2a-b3d2-1f639f3155a5" (UID: "3d0e5840-de72-4c2a-b3d2-1f639f3155a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:57.770878 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770829    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d0e5840-de72-4c2a-b3d2-1f639f3155a5" (UID: "3d0e5840-de72-4c2a-b3d2-1f639f3155a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:57.770878 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770846    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-home" (OuterVolumeSpecName: "home") pod "3d0e5840-de72-4c2a-b3d2-1f639f3155a5" (UID: "3d0e5840-de72-4c2a-b3d2-1f639f3155a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:57.771035 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770926    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-model-cache\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:57.771035 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770942    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tmp-dir\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:57.771035 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.770952    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-home\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:57.772706 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.772674    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-dshm" (OuterVolumeSpecName: "dshm") pod "3d0e5840-de72-4c2a-b3d2-1f639f3155a5" (UID: "3d0e5840-de72-4c2a-b3d2-1f639f3155a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:57.772810 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.772709    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d0e5840-de72-4c2a-b3d2-1f639f3155a5" (UID: "3d0e5840-de72-4c2a-b3d2-1f639f3155a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:43:57.772810 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.772735    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-kube-api-access-kxsfn" (OuterVolumeSpecName: "kube-api-access-kxsfn") pod "3d0e5840-de72-4c2a-b3d2-1f639f3155a5" (UID: "3d0e5840-de72-4c2a-b3d2-1f639f3155a5"). InnerVolumeSpecName "kube-api-access-kxsfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:57.871657 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.871620    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-dshm\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:57.871657 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.871648    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-tls-certs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:57.871657 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:57.871658    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxsfn\" (UniqueName: \"kubernetes.io/projected/3d0e5840-de72-4c2a-b3d2-1f639f3155a5-kube-api-access-kxsfn\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:58.192732 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.192691    2556 generic.go:356] "Generic (PLEG): container finished" podID="3d0e5840-de72-4c2a-b3d2-1f639f3155a5" containerID="4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239" exitCode=0
Jul 13 21:43:58.192924 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.192742    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"
Jul 13 21:43:58.192924 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.192767    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx" event={"ID":"3d0e5840-de72-4c2a-b3d2-1f639f3155a5","Type":"ContainerDied","Data":"4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239"}
Jul 13 21:43:58.192924 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.192804    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx" event={"ID":"3d0e5840-de72-4c2a-b3d2-1f639f3155a5","Type":"ContainerDied","Data":"f2388e1e165098ddab0503e134267b4c84d1a7c454e99e3ad5aecf04f9a677e3"}
Jul 13 21:43:58.192924 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.192827    2556 scope.go:117] "RemoveContainer" containerID="4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239"
Jul 13 21:43:58.203088 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.203071    2556 scope.go:117] "RemoveContainer" containerID="4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239"
Jul 13 21:43:58.203378 ip-10-0-138-146 kubenswrapper[2556]: E0713 21:43:58.203359    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239\": container with ID starting with 4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239 not found: ID does not exist" containerID="4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239"
Jul 13 21:43:58.203441 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.203386    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239"} err="failed to get container status \"4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239\": rpc error: code = NotFound desc = could not find container \"4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239\": container with ID starting with 4c8639eccf0be956dc26a71014a6f6ca1ae8cd0005487aff54694c73ad2b3239 not found: ID does not exist"
Jul 13 21:43:58.217727 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.217692    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"]
Jul 13 21:43:58.226693 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.226660    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-18e186/rate-test-model-18e186-kserve-b74dfc7ff-cs7mx"]
Jul 13 21:43:58.330304 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.330282    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:58.350504 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.350478    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7575-140194763918912_5ed6ea53-b782-4483-96f6-396761d31957/test-curl-7575-140194763918912/0.log"
Jul 13 21:43:58.374948 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.374925    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:58.375180 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.375162    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd9jq\" (UniqueName: \"kubernetes.io/projected/5ed6ea53-b782-4483-96f6-396761d31957-kube-api-access-bd9jq\") pod \"5ed6ea53-b782-4483-96f6-396761d31957\" (UID: \"5ed6ea53-b782-4483-96f6-396761d31957\") "
Jul 13 21:43:58.377584 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.377554    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed6ea53-b782-4483-96f6-396761d31957-kube-api-access-bd9jq" (OuterVolumeSpecName: "kube-api-access-bd9jq") pod "5ed6ea53-b782-4483-96f6-396761d31957" (UID: "5ed6ea53-b782-4483-96f6-396761d31957"). InnerVolumeSpecName "kube-api-access-bd9jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:58.378318 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.378300    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:58.476123 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.476036    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd9jq\" (UniqueName: \"kubernetes.io/projected/5ed6ea53-b782-4483-96f6-396761d31957-kube-api-access-bd9jq\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:43:58.499233 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499207    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:58.499646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499627    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d0e5840-de72-4c2a-b3d2-1f639f3155a5" containerName="main"
Jul 13 21:43:58.499646 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499645    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d0e5840-de72-4c2a-b3d2-1f639f3155a5" containerName="main"
Jul 13 21:43:58.499829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499674    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed6ea53-b782-4483-96f6-396761d31957" containerName="test-curl-7575-140194763918912"
Jul 13 21:43:58.499829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499682    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed6ea53-b782-4483-96f6-396761d31957" containerName="test-curl-7575-140194763918912"
Jul 13 21:43:58.499829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499696    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e83d2b23-578e-40fd-af84-225dff699122" containerName="main"
Jul 13 21:43:58.499829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499703    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e83d2b23-578e-40fd-af84-225dff699122" containerName="main"
Jul 13 21:43:58.499829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499802    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d0e5840-de72-4c2a-b3d2-1f639f3155a5" containerName="main"
Jul 13 21:43:58.499829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499819    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e83d2b23-578e-40fd-af84-225dff699122" containerName="main"
Jul 13 21:43:58.499829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.499828    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed6ea53-b782-4483-96f6-396761d31957" containerName="test-curl-7575-140194763918912"
Jul 13 21:43:58.502965 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.502944    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:58.505951 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.505929    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:58.577431 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.577398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5mzs\" (UniqueName: \"kubernetes.io/projected/58280ec5-eaae-4df0-a102-da1f23c0f76e-kube-api-access-f5mzs\") pod \"test-curl-7575-140194763918912\" (UID: \"58280ec5-eaae-4df0-a102-da1f23c0f76e\") " pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:58.678327 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.678292    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5mzs\" (UniqueName: \"kubernetes.io/projected/58280ec5-eaae-4df0-a102-da1f23c0f76e-kube-api-access-f5mzs\") pod \"test-curl-7575-140194763918912\" (UID: \"58280ec5-eaae-4df0-a102-da1f23c0f76e\") " pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:58.686775 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.686745    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5mzs\" (UniqueName: \"kubernetes.io/projected/58280ec5-eaae-4df0-a102-da1f23c0f76e-kube-api-access-f5mzs\") pod \"test-curl-7575-140194763918912\" (UID: \"58280ec5-eaae-4df0-a102-da1f23c0f76e\") " pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:58.813663 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.813570    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:58.944098 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:58.944072    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:43:58.946024 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:43:58.945990    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58280ec5_eaae_4df0_a102_da1f23c0f76e.slice/crio-74b6ebbe284a721f890f9e15ec239d8993b666506fd81e1b3265d00a6447fd74 WatchSource:0}: Error finding container 74b6ebbe284a721f890f9e15ec239d8993b666506fd81e1b3265d00a6447fd74: Status 404 returned error can't find the container with id 74b6ebbe284a721f890f9e15ec239d8993b666506fd81e1b3265d00a6447fd74
Jul 13 21:43:59.201801 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.201765    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="178ffc4a12c3b91bafcdd7f3afe653af2b6e9e8af1f332ef43d3b47a1bf367ba"
Jul 13 21:43:59.201801 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.201767    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:43:59.202947 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.202925    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763918912" event={"ID":"58280ec5-eaae-4df0-a102-da1f23c0f76e","Type":"ContainerStarted","Data":"74b6ebbe284a721f890f9e15ec239d8993b666506fd81e1b3265d00a6447fd74"}
Jul 13 21:43:59.213433 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.213408    2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7575-140194763918912" oldPodUID="5ed6ea53-b782-4483-96f6-396761d31957" podUID="58280ec5-eaae-4df0-a102-da1f23c0f76e"
Jul 13 21:43:59.603660 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.603583    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d0e5840-de72-4c2a-b3d2-1f639f3155a5" path="/var/lib/kubelet/pods/3d0e5840-de72-4c2a-b3d2-1f639f3155a5/volumes"
Jul 13 21:43:59.603908 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.603895    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ed6ea53-b782-4483-96f6-396761d31957" path="/var/lib/kubelet/pods/5ed6ea53-b782-4483-96f6-396761d31957/volumes"
Jul 13 21:43:59.804530 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.804499    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:43:59.813679 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:43:59.813659    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:44:00.210748 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:00.210715    2556 generic.go:356] "Generic (PLEG): container finished" podID="58280ec5-eaae-4df0-a102-da1f23c0f76e" containerID="7aa66a195c659d3a064afcd7b8db3d17f31a788bde6ec2e302a1f40095847d7c" exitCode=0
Jul 13 21:44:00.210918 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:00.210752    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7575-140194763918912" event={"ID":"58280ec5-eaae-4df0-a102-da1f23c0f76e","Type":"ContainerDied","Data":"7aa66a195c659d3a064afcd7b8db3d17f31a788bde6ec2e302a1f40095847d7c"}
Jul 13 21:44:01.358916 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.358891    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:44:01.380881 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.380851    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7575-140194763918912_58280ec5-eaae-4df0-a102-da1f23c0f76e/test-curl-7575-140194763918912/0.log"
Jul 13 21:44:01.409514 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.409481    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:44:01.413067 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.413045    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7575-140194763918912"]
Jul 13 21:44:01.503588 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.503517    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5mzs\" (UniqueName: \"kubernetes.io/projected/58280ec5-eaae-4df0-a102-da1f23c0f76e-kube-api-access-f5mzs\") pod \"58280ec5-eaae-4df0-a102-da1f23c0f76e\" (UID: \"58280ec5-eaae-4df0-a102-da1f23c0f76e\") "
Jul 13 21:44:01.505575 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.505549    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58280ec5-eaae-4df0-a102-da1f23c0f76e-kube-api-access-f5mzs" (OuterVolumeSpecName: "kube-api-access-f5mzs") pod "58280ec5-eaae-4df0-a102-da1f23c0f76e" (UID: "58280ec5-eaae-4df0-a102-da1f23c0f76e"). InnerVolumeSpecName "kube-api-access-f5mzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:44:01.603125 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.603091    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58280ec5-eaae-4df0-a102-da1f23c0f76e" path="/var/lib/kubelet/pods/58280ec5-eaae-4df0-a102-da1f23c0f76e/volumes"
Jul 13 21:44:01.604563 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:01.604542    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5mzs\" (UniqueName: \"kubernetes.io/projected/58280ec5-eaae-4df0-a102-da1f23c0f76e-kube-api-access-f5mzs\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:44:02.223623 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:02.223596    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7575-140194763918912"
Jul 13 21:44:02.223623 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:02.223615    2556 scope.go:117] "RemoveContainer" containerID="7aa66a195c659d3a064afcd7b8db3d17f31a788bde6ec2e302a1f40095847d7c"
Jul 13 21:44:15.942132 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:15.942100    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:44:27.729042 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.729008    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"]
Jul 13 21:44:27.729443 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.729432    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58280ec5-eaae-4df0-a102-da1f23c0f76e" containerName="test-curl-7575-140194763918912"
Jul 13 21:44:27.729507 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.729446    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58280ec5-eaae-4df0-a102-da1f23c0f76e" containerName="test-curl-7575-140194763918912"
Jul 13 21:44:27.729543 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.729516    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58280ec5-eaae-4df0-a102-da1f23c0f76e" containerName="test-curl-7575-140194763918912"
Jul 13 21:44:27.734417 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.734401    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"
Jul 13 21:44:27.737545 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.737518    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rnndk\""
Jul 13 21:44:27.737677 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.737523    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:44:27.739705 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.739684    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"]
Jul 13 21:44:27.742961 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.742582    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk\" (UID: \"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"
Jul 13 21:44:27.843861 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.843822    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk\" (UID: \"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"
Jul 13 21:44:27.844507 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:27.844486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk\" (UID: \"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"
Jul 13 21:44:28.047806 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:28.047719    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"
Jul 13 21:44:28.199619 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:28.199590    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"]
Jul 13 21:44:28.200638 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:44:28.200615    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd190f1f0_7c5a_4918_8c69_34ba6cfaa8c1.slice/crio-4ec5f1b2a11a8da218fde8b19654231fbb6f56e43ef0fd88209387a15f20d0d1 WatchSource:0}: Error finding container 4ec5f1b2a11a8da218fde8b19654231fbb6f56e43ef0fd88209387a15f20d0d1: Status 404 returned error can't find the container with id 4ec5f1b2a11a8da218fde8b19654231fbb6f56e43ef0fd88209387a15f20d0d1
Jul 13 21:44:28.372637 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:28.372602    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk" event={"ID":"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1","Type":"ContainerStarted","Data":"4ec5f1b2a11a8da218fde8b19654231fbb6f56e43ef0fd88209387a15f20d0d1"}
Jul 13 21:44:29.379454 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:29.379414    2556 generic.go:356] "Generic (PLEG): container finished" podID="d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1" containerID="0a15a6730089736fc11db1864ad045c5f4627243401771ad4512163dab384aed" exitCode=0
Jul 13 21:44:29.379824 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:29.379460    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk" event={"ID":"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1","Type":"ContainerDied","Data":"0a15a6730089736fc11db1864ad045c5f4627243401771ad4512163dab384aed"}
Jul 13 21:44:30.526192 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:30.526167    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"
Jul 13 21:44:30.566958 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:30.566928    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1-maas-api-service-ca\") pod \"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1\" (UID: \"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1\") "
Jul 13 21:44:30.567276 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:30.567255    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1" (UID: "d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:44:30.668355 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:30.668258    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1-maas-api-service-ca\") on node \"ip-10-0-138-146.ec2.internal\" DevicePath \"\""
Jul 13 21:44:31.392460 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:31.392429    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk"
Jul 13 21:44:31.392634 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:31.392461    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk" event={"ID":"d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1","Type":"ContainerDied","Data":"4ec5f1b2a11a8da218fde8b19654231fbb6f56e43ef0fd88209387a15f20d0d1"}
Jul 13 21:44:31.392634 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:31.392537    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ec5f1b2a11a8da218fde8b19654231fbb6f56e43ef0fd88209387a15f20d0d1"
Jul 13 21:44:41.538435 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:41.538402    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sb5qt"]
Jul 13 21:44:54.156427 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:54.156316    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f7b5d5cfc-449kl_c21efc3d-4ebc-472b-a4d7-5546b51ad755/authorino/0.log"
Jul 13 21:44:59.818431 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:59.818401    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fdc557c48-j4x96_f53550a6-a891-491c-b81e-2a39d2713b6c/copy-manifests/0.log"
Jul 13 21:44:59.824755 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:44:59.824728    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fdc557c48-j4x96_f53550a6-a891-491c-b81e-2a39d2713b6c/manager/0.log"
Jul 13 21:45:00.894201 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:00.894177    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-19927e2d-fhmqm_9d9de8c3-fcc6-4664-ab88-160cc1e5dac0/revoke-keys/0.log"
Jul 13 21:45:01.551331 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:01.551296    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-2a506c-m2gg8_d69882e5-858a-45a9-bc11-b7b1711b2fb8/revoke-keys/0.log"
Jul 13 21:45:02.116551 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:02.116518    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-18e186-ztzkk_d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1/revoke-keys/0.log"
Jul 13 21:45:02.989628 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:02.989595    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm_b543c256-f9a0-42c0-8dbc-f297fd076faf/util/0.log"
Jul 13 21:45:02.998206 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:02.998167    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm_b543c256-f9a0-42c0-8dbc-f297fd076faf/pull/0.log"
Jul 13 21:45:03.005441 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.005415    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm_b543c256-f9a0-42c0-8dbc-f297fd076faf/extract/0.log"
Jul 13 21:45:03.110993 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.110958    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf_e505efe2-5353-4072-a357-38c172513a8f/util/0.log"
Jul 13 21:45:03.116571 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.116548    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf_e505efe2-5353-4072-a357-38c172513a8f/pull/0.log"
Jul 13 21:45:03.121682 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.121663    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf_e505efe2-5353-4072-a357-38c172513a8f/extract/0.log"
Jul 13 21:45:03.226281 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.226255    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp_7b775ec9-cc04-4ccf-9738-cb241f66886d/util/0.log"
Jul 13 21:45:03.232356 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.232317    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp_7b775ec9-cc04-4ccf-9738-cb241f66886d/pull/0.log"
Jul 13 21:45:03.237992 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.237971    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp_7b775ec9-cc04-4ccf-9738-cb241f66886d/extract/0.log"
Jul 13 21:45:03.347829 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.347754    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75_f36704e8-6638-4cd9-8fa8-a6a10512ba21/extract/0.log"
Jul 13 21:45:03.353440 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.353414    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75_f36704e8-6638-4cd9-8fa8-a6a10512ba21/util/0.log"
Jul 13 21:45:03.359555 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.359529    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75_f36704e8-6638-4cd9-8fa8-a6a10512ba21/pull/0.log"
Jul 13 21:45:03.477534 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.477508    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f7b5d5cfc-449kl_c21efc3d-4ebc-472b-a4d7-5546b51ad755/authorino/0.log"
Jul 13 21:45:03.811888 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:03.811860    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7cgn9_e51b2976-ac19-413c-aab1-c35e6096060c/kuadrant-console-plugin/0.log"
Jul 13 21:45:04.183103 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:04.183079    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sb5qt_d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a/limitador/0.log"
Jul 13 21:45:04.976007 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:04.975977    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf978c8f5-dbx8w_61856d71-02cf-46e3-8918-77a9df987dcb/kube-auth-proxy/0.log"
Jul 13 21:45:05.447762 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:05.447726    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7488d7cbb8-nn48c_f0172f3d-38f3-4259-88f8-258f6424f8e0/router/0.log"
Jul 13 21:45:06.364238 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:06.364199    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-54n8m_01b0f5bd-b744-4f07-b2d3-e5b8354f252c/main/0.log"
Jul 13 21:45:10.152825 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.152779    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sxm8t/must-gather-v4286"]
Jul 13 21:45:10.153269 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.153249    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1" containerName="revoke-keys"
Jul 13 21:45:10.153356 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.153273    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1" containerName="revoke-keys"
Jul 13 21:45:10.153414 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.153380    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d190f1f0-7c5a-4918-8c69-34ba6cfaa8c1" containerName="revoke-keys"
Jul 13 21:45:10.156655 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.156637    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.159012 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.158992    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sxm8t\"/\"kube-root-ca.crt\""
Jul 13 21:45:10.159958 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.159935    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sxm8t\"/\"default-dockercfg-4dwcg\""
Jul 13 21:45:10.160028 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.159939    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sxm8t\"/\"openshift-service-ca.crt\""
Jul 13 21:45:10.166503 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.166483    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxm8t/must-gather-v4286"]
Jul 13 21:45:10.226567 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.226533    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1ad32911-f1b2-4bb8-9a14-d6294ad2d36e-must-gather-output\") pod \"must-gather-v4286\" (UID: \"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e\") " pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.226719 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.226577    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5ntk\" (UniqueName: \"kubernetes.io/projected/1ad32911-f1b2-4bb8-9a14-d6294ad2d36e-kube-api-access-f5ntk\") pod \"must-gather-v4286\" (UID: \"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e\") " pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.327102 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.327066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1ad32911-f1b2-4bb8-9a14-d6294ad2d36e-must-gather-output\") pod \"must-gather-v4286\" (UID: \"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e\") " pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.327257 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.327115    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5ntk\" (UniqueName: \"kubernetes.io/projected/1ad32911-f1b2-4bb8-9a14-d6294ad2d36e-kube-api-access-f5ntk\") pod \"must-gather-v4286\" (UID: \"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e\") " pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.327424 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.327405    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1ad32911-f1b2-4bb8-9a14-d6294ad2d36e-must-gather-output\") pod \"must-gather-v4286\" (UID: \"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e\") " pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.335170 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.335150    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5ntk\" (UniqueName: \"kubernetes.io/projected/1ad32911-f1b2-4bb8-9a14-d6294ad2d36e-kube-api-access-f5ntk\") pod \"must-gather-v4286\" (UID: \"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e\") " pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.465812 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.465736    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxm8t/must-gather-v4286"
Jul 13 21:45:10.596705 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.596671    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxm8t/must-gather-v4286"]
Jul 13 21:45:10.597831 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:45:10.597802    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ad32911_f1b2_4bb8_9a14_d6294ad2d36e.slice/crio-72aea90c3a359e964352fe195015076efc8ec035c1ef88c9f95fbcad4b355d08 WatchSource:0}: Error finding container 72aea90c3a359e964352fe195015076efc8ec035c1ef88c9f95fbcad4b355d08: Status 404 returned error can't find the container with id 72aea90c3a359e964352fe195015076efc8ec035c1ef88c9f95fbcad4b355d08
Jul 13 21:45:10.612708 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:10.612676    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxm8t/must-gather-v4286" event={"ID":"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e","Type":"ContainerStarted","Data":"72aea90c3a359e964352fe195015076efc8ec035c1ef88c9f95fbcad4b355d08"}
Jul 13 21:45:11.623272 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:11.623230    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxm8t/must-gather-v4286" event={"ID":"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e","Type":"ContainerStarted","Data":"19a978b8323f16bb84aa086825c2d13cd60ade87a015fd3afdaa3e1d64712b5e"}
Jul 13 21:45:11.623762 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:11.623283    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxm8t/must-gather-v4286" event={"ID":"1ad32911-f1b2-4bb8-9a14-d6294ad2d36e","Type":"ContainerStarted","Data":"157c887fa14cf1b0424fecf784aecd2eb79625d675586dfda872aaab47af9e62"}
Jul 13 21:45:11.641773 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:11.641663    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sxm8t/must-gather-v4286" podStartSLOduration=0.849587856 podStartE2EDuration="1.641645618s" totalImagesPullingTime="792.057762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:45:11.641157348 +0000 UTC m=+3072.579361683" watchObservedRunningTime="2026-07-13 21:45:11.641645618 +0000 UTC m=+3072.579849931"
Jul 13 21:45:13.071592 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:13.071564    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2z72g_0420276d-4854-4250-9a5d-5ebdfad81a34/global-pull-secret-syncer/0.log"
Jul 13 21:45:13.234040 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:13.234009    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9kv8m_f457d97c-0c11-4379-ac4a-6d8033a4cff7/konnectivity-agent/0.log"
Jul 13 21:45:13.336315 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:13.336236    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-146.ec2.internal_d10ad1489d9712613b909680a7e53bfb/haproxy/0.log"
Jul 13 21:45:17.028278 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.028244    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm_b543c256-f9a0-42c0-8dbc-f297fd076faf/extract/0.log"
Jul 13 21:45:17.063112 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.063084    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm_b543c256-f9a0-42c0-8dbc-f297fd076faf/util/0.log"
Jul 13 21:45:17.099324 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.099295    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bc6bm_b543c256-f9a0-42c0-8dbc-f297fd076faf/pull/0.log"
Jul 13 21:45:17.138227 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.138199    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf_e505efe2-5353-4072-a357-38c172513a8f/extract/0.log"
Jul 13 21:45:17.162912 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.162888    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf_e505efe2-5353-4072-a357-38c172513a8f/util/0.log"
Jul 13 21:45:17.189255 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.189227    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsstf_e505efe2-5353-4072-a357-38c172513a8f/pull/0.log"
Jul 13 21:45:17.217692 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.217665    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp_7b775ec9-cc04-4ccf-9738-cb241f66886d/extract/0.log"
Jul 13 21:45:17.245832 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.245807    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp_7b775ec9-cc04-4ccf-9738-cb241f66886d/util/0.log"
Jul 13 21:45:17.276286 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.276259    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jq8lp_7b775ec9-cc04-4ccf-9738-cb241f66886d/pull/0.log"
Jul 13 21:45:17.317959 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.317851    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75_f36704e8-6638-4cd9-8fa8-a6a10512ba21/extract/0.log"
Jul 13 21:45:17.346564 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.346538    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75_f36704e8-6638-4cd9-8fa8-a6a10512ba21/util/0.log"
Jul 13 21:45:17.368607 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.368578    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1trn75_f36704e8-6638-4cd9-8fa8-a6a10512ba21/pull/0.log"
Jul 13 21:45:17.890371 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.890296    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f7b5d5cfc-449kl_c21efc3d-4ebc-472b-a4d7-5546b51ad755/authorino/0.log"
Jul 13 21:45:17.994184 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:17.994158    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7cgn9_e51b2976-ac19-413c-aab1-c35e6096060c/kuadrant-console-plugin/0.log"
Jul 13 21:45:18.229840 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:18.229752    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sb5qt_d9197f55-c9c8-4c47-a9ca-c9e2d2a4132a/limitador/0.log"
Jul 13 21:45:19.810782 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.810749    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_53590627-1cd2-43e9-a8bb-fc13ae7acf7a/alertmanager/0.log"
Jul 13 21:45:19.832094 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.832055    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_53590627-1cd2-43e9-a8bb-fc13ae7acf7a/config-reloader/0.log"
Jul 13 21:45:19.853690 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.853665    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_53590627-1cd2-43e9-a8bb-fc13ae7acf7a/kube-rbac-proxy-web/0.log"
Jul 13 21:45:19.874019 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.873992    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_53590627-1cd2-43e9-a8bb-fc13ae7acf7a/kube-rbac-proxy/0.log"
Jul 13 21:45:19.894291 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.894264    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_53590627-1cd2-43e9-a8bb-fc13ae7acf7a/kube-rbac-proxy-metric/0.log"
Jul 13 21:45:19.914195 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.914157    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_53590627-1cd2-43e9-a8bb-fc13ae7acf7a/prom-label-proxy/0.log"
Jul 13 21:45:19.934134 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.934102    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_53590627-1cd2-43e9-a8bb-fc13ae7acf7a/init-config-reloader/0.log"
Jul 13 21:45:19.977097 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:19.977067    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-2vxxv_90d21d66-81d6-464c-a857-7d43f52beb1d/cluster-monitoring-operator/0.log"
Jul 13 21:45:20.344475 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.344365    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgzt9_acefebc3-f576-4baa-9913-d83d1e381678/node-exporter/0.log"
Jul 13 21:45:20.376555 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.376524    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgzt9_acefebc3-f576-4baa-9913-d83d1e381678/kube-rbac-proxy/0.log"
Jul 13 21:45:20.400093 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.400018    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgzt9_acefebc3-f576-4baa-9913-d83d1e381678/init-textfile/0.log"
Jul 13 21:45:20.441534 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.441496    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-c97lj_9f1c8eac-1876-4340-8b83-feccc327e79a/kube-rbac-proxy-main/0.log"
Jul 13 21:45:20.461413 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.461385    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-c97lj_9f1c8eac-1876-4340-8b83-feccc327e79a/kube-rbac-proxy-self/0.log"
Jul 13 21:45:20.482579 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.482550    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-c97lj_9f1c8eac-1876-4340-8b83-feccc327e79a/openshift-state-metrics/0.log"
Jul 13 21:45:20.740847 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.740765    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vqpcz_89757365-9849-4ec4-b600-5480a74b265e/prometheus-operator/0.log"
Jul 13 21:45:20.756894 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.756864    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vqpcz_89757365-9849-4ec4-b600-5480a74b265e/kube-rbac-proxy/0.log"
Jul 13 21:45:20.818921 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.818887    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5696968dc5-nxh7z_c0156185-4b50-4b11-9332-fd09471fa634/telemeter-client/0.log"
Jul 13 21:45:20.843804 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.843773    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5696968dc5-nxh7z_c0156185-4b50-4b11-9332-fd09471fa634/reload/0.log"
Jul 13 21:45:20.866730 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.866692    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5696968dc5-nxh7z_c0156185-4b50-4b11-9332-fd09471fa634/kube-rbac-proxy/0.log"
Jul 13 21:45:20.899779 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.899748    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5499b7b54d-qgj47_ae2bb4da-7bcb-4ea8-bed9-387890bc3f24/thanos-query/0.log"
Jul 13 21:45:20.921473 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.921442    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5499b7b54d-qgj47_ae2bb4da-7bcb-4ea8-bed9-387890bc3f24/kube-rbac-proxy-web/0.log"
Jul 13 21:45:20.942853 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.942819    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5499b7b54d-qgj47_ae2bb4da-7bcb-4ea8-bed9-387890bc3f24/kube-rbac-proxy/0.log"
Jul 13 21:45:20.966697 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.966668    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5499b7b54d-qgj47_ae2bb4da-7bcb-4ea8-bed9-387890bc3f24/prom-label-proxy/0.log"
Jul 13 21:45:20.988050 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:20.988022    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5499b7b54d-qgj47_ae2bb4da-7bcb-4ea8-bed9-387890bc3f24/kube-rbac-proxy-rules/0.log"
Jul 13 21:45:21.011840 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:21.011764    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5499b7b54d-qgj47_ae2bb4da-7bcb-4ea8-bed9-387890bc3f24/kube-rbac-proxy-metrics/0.log"
Jul 13 21:45:22.851056 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:22.851032    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/1.log"
Jul 13 21:45:22.860368 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:22.860305    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gd4wz_0cd5c58a-b847-450e-aebd-3a21b97e4f10/console-operator/2.log"
Jul 13 21:45:23.252166 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.252125    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"]
Jul 13 21:45:23.260563 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.260537    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.275642 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.271263    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"]
Jul 13 21:45:23.385933 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.385896    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-podres\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.386215 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.386200    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-proc\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.386380 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.386362    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-sys\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.386566 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.386552    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-lib-modules\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.386698 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.386686    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-774q4\" (UniqueName: \"kubernetes.io/projected/1d0f7afc-7514-45aa-b0a6-989263c5780a-kube-api-access-774q4\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488044 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488012    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-podres\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488224 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488054    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-proc\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488224 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488084    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-sys\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488224 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488149    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-lib-modules\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488224 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488182    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-774q4\" (UniqueName: \"kubernetes.io/projected/1d0f7afc-7514-45aa-b0a6-989263c5780a-kube-api-access-774q4\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488459 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488217    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-podres\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488459 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488355    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-sys\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488459 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488410    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-proc\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.488609 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.488506    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d0f7afc-7514-45aa-b0a6-989263c5780a-lib-modules\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.496787 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.496756    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-774q4\" (UniqueName: \"kubernetes.io/projected/1d0f7afc-7514-45aa-b0a6-989263c5780a-kube-api-access-774q4\") pod \"perf-node-gather-daemonset-mrn9m\" (UID: \"1d0f7afc-7514-45aa-b0a6-989263c5780a\") " pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.587703 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.587612    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:23.775084 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:23.772426    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"]
Jul 13 21:45:23.775745 ip-10-0-138-146 kubenswrapper[2556]: W0713 21:45:23.775507    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1d0f7afc_7514_45aa_b0a6_989263c5780a.slice/crio-5cd045ab7c6789be3b4600d399cee7f5f182de7751d4c6f06825e8c9cd0b4e8a WatchSource:0}: Error finding container 5cd045ab7c6789be3b4600d399cee7f5f182de7751d4c6f06825e8c9cd0b4e8a: Status 404 returned error can't find the container with id 5cd045ab7c6789be3b4600d399cee7f5f182de7751d4c6f06825e8c9cd0b4e8a
Jul 13 21:45:24.166361 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:24.166304    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-nqddc_97d7cdc8-2942-4d1a-a64e-aaef2f94dbc2/volume-data-source-validator/0.log"
Jul 13 21:45:24.781293 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:24.781261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m" event={"ID":"1d0f7afc-7514-45aa-b0a6-989263c5780a","Type":"ContainerStarted","Data":"98802b7fbea3140d4ee30d93ad4dbe05a6698c067b0b90b52006ad79dafc6ea8"}
Jul 13 21:45:24.781484 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:24.781313    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m" event={"ID":"1d0f7afc-7514-45aa-b0a6-989263c5780a","Type":"ContainerStarted","Data":"5cd045ab7c6789be3b4600d399cee7f5f182de7751d4c6f06825e8c9cd0b4e8a"}
Jul 13 21:45:24.781484 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:24.781387    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:24.798506 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:24.798458    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m" podStartSLOduration=1.7984434569999999 podStartE2EDuration="1.798443457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:45:24.797959824 +0000 UTC m=+3085.736164162" watchObservedRunningTime="2026-07-13 21:45:24.798443457 +0000 UTC m=+3085.736647808"
Jul 13 21:45:25.268923 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:25.268891    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4fxfq_326c721f-7a82-4669-b58c-153c8ac68f50/dns/0.log"
Jul 13 21:45:25.288295 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:25.288273    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4fxfq_326c721f-7a82-4669-b58c-153c8ac68f50/kube-rbac-proxy/0.log"
Jul 13 21:45:25.357383 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:25.357333    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n688c_38b9c8cd-9d66-430c-b411-e92d0a730deb/dns-node-resolver/0.log"
Jul 13 21:45:26.031695 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:26.031661    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9pjnw_3ba71b3d-1630-4cd4-912c-2ed8f84aad46/node-ca/0.log"
Jul 13 21:45:27.120357 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:27.120305    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf978c8f5-dbx8w_61856d71-02cf-46e3-8918-77a9df987dcb/kube-auth-proxy/0.log"
Jul 13 21:45:27.262289 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:27.262237    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7488d7cbb8-nn48c_f0172f3d-38f3-4259-88f8-258f6424f8e0/router/0.log"
Jul 13 21:45:27.992327 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:27.992295    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7gknj_20d69c93-0f84-46ed-95ab-2d19b1515e04/serve-healthcheck-canary/0.log"
Jul 13 21:45:28.506020 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:28.505988    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-mvx2z_7518b6cb-fdc2-46b6-934c-e73079d7a0d2/insights-operator/0.log"
Jul 13 21:45:28.506936 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:28.506914    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-mvx2z_7518b6cb-fdc2-46b6-934c-e73079d7a0d2/insights-operator/1.log"
Jul 13 21:45:28.663956 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:28.663929    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w8tqk_6ee9564f-ce55-439b-b58c-89c8721bd217/kube-rbac-proxy/0.log"
Jul 13 21:45:28.683397 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:28.683370    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w8tqk_6ee9564f-ce55-439b-b58c-89c8721bd217/exporter/0.log"
Jul 13 21:45:28.704037 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:28.704014    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w8tqk_6ee9564f-ce55-439b-b58c-89c8721bd217/extractor/0.log"
Jul 13 21:45:30.799961 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:30.799932    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sxm8t/perf-node-gather-daemonset-mrn9m"
Jul 13 21:45:31.247419 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:31.247392    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fdc557c48-j4x96_f53550a6-a891-491c-b81e-2a39d2713b6c/manager/0.log"
Jul 13 21:45:31.268528 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:31.268499    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fdc557c48-j4x96_f53550a6-a891-491c-b81e-2a39d2713b6c/copy-manifests/0.log"
Jul 13 21:45:32.754613 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:32.754570    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f779bff78-r7hg4_67fef059-1701-44e2-9868-017589abe5b1/manager/0.log"
Jul 13 21:45:32.811711 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:32.811629    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-m47nk_3020b304-ddbc-4aae-9769-40fbecd67cb4/openshift-lws-operator/0.log"
Jul 13 21:45:33.788054 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:33.787972    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf978c8f5-dbx8w_61856d71-02cf-46e3-8918-77a9df987dcb/kube-auth-proxy/0.log"
Jul 13 21:45:33.897082 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:33.897053    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7488d7cbb8-nn48c_f0172f3d-38f3-4259-88f8-258f6424f8e0/router/0.log"
Jul 13 21:45:35.319204 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:35.319176    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bf978c8f5-dbx8w_61856d71-02cf-46e3-8918-77a9df987dcb/kube-auth-proxy/0.log"
Jul 13 21:45:35.430075 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:35.430029    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7488d7cbb8-nn48c_f0172f3d-38f3-4259-88f8-258f6424f8e0/router/0.log"
Jul 13 21:45:38.379796 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:38.379763    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bkql9_f7e89441-6447-4b44-b1e3-b3a8ead9aa13/kube-storage-version-migrator-operator/1.log"
Jul 13 21:45:38.381565 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:38.381532    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bkql9_f7e89441-6447-4b44-b1e3-b3a8ead9aa13/kube-storage-version-migrator-operator/0.log"
Jul 13 21:45:39.744500 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.744472    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmwk4_fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2/kube-multus-additional-cni-plugins/0.log"
Jul 13 21:45:39.765631 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.765606    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmwk4_fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2/egress-router-binary-copy/0.log"
Jul 13 21:45:39.784331 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.784293    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmwk4_fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2/cni-plugins/0.log"
Jul 13 21:45:39.804586 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.804560    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmwk4_fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2/bond-cni-plugin/0.log"
Jul 13 21:45:39.823960 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.823939    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmwk4_fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2/routeoverride-cni/0.log"
Jul 13 21:45:39.843683 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.843663    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmwk4_fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2/whereabouts-cni-bincopy/0.log"
Jul 13 21:45:39.862323 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.862292    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zmwk4_fa8cfbe1-0c58-4bb5-94b9-1d536f62aaf2/whereabouts-cni/0.log"
Jul 13 21:45:39.905183 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:39.905160    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lv4m5_21e605a3-44ba-43d8-b5e5-4e365565ed28/kube-multus/0.log"
Jul 13 21:45:40.029144 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:40.029063    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-grcrp_de4035ca-3984-435e-8387-217677bcc954/network-metrics-daemon/0.log"
Jul 13 21:45:40.046375 ip-10-0-138-146 kubenswrapper[2556]: I0713 21:45:40.046349    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-grcrp_de4035ca-3984-435e-8387-217677bcc954/kube-rbac-proxy/0.log"