Jul 07 13:13:34.870092 ip-10-0-128-146 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.305948    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311910    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311922    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311926    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311930    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311932    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311935    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311938    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:13:35.455412 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311942    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311945    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311948    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311954    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311958    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311962    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311966    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311969    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311971    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311975    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311977    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311980    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311983    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311986    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311988    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311991    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311994    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311997    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.311999    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:13:35.605064 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312002    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:13:35.380986 ip-10-0-128-146 systemd[1]: Started Kubernetes Kubelet.
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312004    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312007    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312010    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312013    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312016    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312019    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312021    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312026    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312029    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312032    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312035    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312038    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312041    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312043    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312046    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312049    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312051    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312054    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:13:35.607141 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312057    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312059    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312064    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312068    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312071    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312074    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312076    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312079    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312081    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312084    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312086    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312089    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312092    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312094    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312097    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312100    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312102    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312107    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312109    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312113    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:13:35.609676 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312116    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312119    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312121    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312124    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312126    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312129    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312132    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312134    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312137    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312139    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312142    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312145    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312147    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312150    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312152    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312155    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312158    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312160    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312163    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312165    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:13:35.611789 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312168    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312174    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312176    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312179    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312181    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312184    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312187    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312189    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312193    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312196    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312198    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312201    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312204    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312206    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312926    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312933    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312937    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312939    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312945    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312948    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:13:35.614222 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312951    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312953    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312956    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312959    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312962    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312965    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312967    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312970    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312973    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312976    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312979    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312982    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312984    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312987    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312990    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312992    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312995    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.312998    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313000    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313003    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:13:35.616273 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313007    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313010    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313012    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313015    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313018    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313021    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313024    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313026    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313030    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313034    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313038    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313041    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313044    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313047    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313049    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313052    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313054    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313057    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313059    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:13:35.618962 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313062    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313064    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313067    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313069    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313072    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313075    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313077    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313080    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313082    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313087    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313091    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313094    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313097    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313101    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313104    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313107    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313110    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313113    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313115    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:13:35.621246 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313118    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313121    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313124    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313127    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313131    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313133    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313136    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313139    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313141    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313144    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313146    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313149    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313152    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313154    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313157    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313160    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313163    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313165    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313168    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313171    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:13:35.624522 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313173    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313176    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313178    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313181    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313183    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313186    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313189    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313193    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313197    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313200    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313221    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313224    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313228    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313231    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313234    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313583    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313589    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313592    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313595    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:13:35.627550 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313597    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313600    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313602    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313605    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313608    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313610    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313613    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313616    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313618    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313621    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313624    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313626    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313629    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313631    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313634    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313636    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313639    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313642    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313644    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313649    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:13:35.630758 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313651    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313654    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313656    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313659    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313661    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313664    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313667    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313670    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313672    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313676    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313679    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313681    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313684    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313686    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313689    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313691    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313694    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313696    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313699    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313701    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:13:35.634732 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313704    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313706    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313710    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313714    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313718    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313722    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313725    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313728    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313730    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313733    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313736    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313741    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313743    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313746    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313749    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313751    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313754    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313757    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313760    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:13:35.638111 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313763    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313765    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313769    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313772    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313775    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313777    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313779    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313782    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313784    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313787    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313789    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313792    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313795    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313798    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313801    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313803    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313805    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313823    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313826    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313829    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:13:35.639317 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313832    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313834    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313837    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313839    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313843    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313846    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313849    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313851    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313854    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313856    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313859    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313862    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313864    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313867    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313873    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.313876    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313941    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313952    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313958    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313963    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313967    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 13:13:35.640438 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313970    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313974    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313978    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313981    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313984    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313987    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313991    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313994    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.313997    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314000    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314003    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314006    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314009    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314013    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314016    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314020    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314024    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314027    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314030    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314033    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314036    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314040    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314043    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314046    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 13:13:35.641230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314049    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314054    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314057    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314060    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314063    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314066    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314069    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314072    2577 flags.go:64] FLAG: --enable-server="true"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314075    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314079    2577 flags.go:64] FLAG: --event-burst="100"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314083    2577 flags.go:64] FLAG: --event-qps="50"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314086    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314089    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314091    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314095    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314098    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314101    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314104    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314107    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314109    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314112    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314115    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314118    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314122    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314125    2577 flags.go:64] FLAG: --feature-gates=""
Jul 07 13:13:35.642017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314129    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314132    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314135    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314138    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314141    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314143    2577 flags.go:64] FLAG: --help="false"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314146    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-128-146.ec2.internal"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314149    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314153    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314157    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314160    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314163    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314166    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314169    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314172    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314174    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314178    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314181    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314183    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314186    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314189    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314192    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314195    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314197    2577 flags.go:64] FLAG: --lock-file=""
Jul 07 13:13:35.642741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314200    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314203    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314206    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314211    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314214    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314217    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314221    2577 flags.go:64] FLAG: --logging-format="text"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314224    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314228    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314230    2577 flags.go:64] FLAG: --manifest-url=""
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314233    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314237    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314240    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314244    2577 flags.go:64] FLAG: --max-pods="110"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314247    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314250    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314254    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314257    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314261    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314264    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314267    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314273    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314276    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314279    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 13:13:35.643483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314282    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314285    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314290    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314293    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314296    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314299    2577 flags.go:64] FLAG: --port="10250"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314302    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314305    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-011f03874ca2430f4"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314308    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314311    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314314    2577 flags.go:64] FLAG: --register-node="true"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314317    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314320    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314323    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314326    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314331    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314335    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314338    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314341    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314343    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314346    2577 flags.go:64] FLAG: --runonce="false"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314349    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314352    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314355    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314359    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 13:13:35.644524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314362    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314364    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314368    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314371    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314374    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314376    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314379    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314382    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314385    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314388    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314391    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314396    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314399    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314402    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314406    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314408    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314411    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314414    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314417    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314419    2577 flags.go:64] FLAG: --v="2"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314423    2577 flags.go:64] FLAG: --version="false"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314427    2577 flags.go:64] FLAG: --vmodule=""
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314433    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314436    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314541    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:13:35.645251 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314545    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314550    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314553    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314557    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314559    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314562    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314567    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314570    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314573    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314575    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314579    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314582    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314584    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314587    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314589    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314592    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314595    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314598    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:13:35.882257 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314601    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314605    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314608    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314610    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314613    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314616    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314618    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314621    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314624    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314626    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314629    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314633    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314635    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314638    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314640    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314643    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314645    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314648    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314650    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314653    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:13:35.883067 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314657    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314660    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314663    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314665    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314668    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314671    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314673    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314676    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314678    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314681    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314683    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314686    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314689    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314691    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314694    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314696    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314698    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314701    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314704    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314706    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:13:35.883619 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314709    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314711    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314714    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314717    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314720    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314722    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314725    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314727    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314730    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314733    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314735    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314738    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314742    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314744    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314747    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314749    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314752    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314754    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314758    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314760    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:13:35.884264 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314763    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314765    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314768    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314770    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314773    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314776    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314778    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314781    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314783    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314786    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314788    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314790    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314793    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314796    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314798    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314802    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314806    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314823    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314827    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:13:35.884783 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.314831    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.314862    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.323193    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.323209    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.323228    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.323238    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323294    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323300    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323304    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323307    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323310    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323312    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323315    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323318    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323320    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:13:35.885316 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323323    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323326    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323330    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323335    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323338    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323341    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323344    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323347    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323351    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323355    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323358    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323360    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323366    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323368    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323372    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323375    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323378    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323381    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323385    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323387    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:13:35.885734 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323390    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323393    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323395    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323398    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323401    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323404    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323406    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323409    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323412    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323414    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323417    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323420    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323423    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323425    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323428    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323431    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323434    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323436    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323439    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:13:35.886300 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323442    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323445    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323448    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323450    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323453    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323459    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323462    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323466    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323469    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323472    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323474    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323478    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323481    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323484    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323486    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323489    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323492    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323494    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323497    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:13:35.886793 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323500    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323502    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323505    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323507    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323510    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323513    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323515    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323518    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323521    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323523    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323526    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323529    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323533    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323535    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323538    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323540    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323543    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323546    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323550    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323552    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:13:35.887336 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323555    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323558    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323560    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323563    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323566    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323568    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323571    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323573    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323576    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323578    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323581    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323584    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.323589    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323735    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323740    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 13:13:35.887904 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323743    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323746    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323749    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323752    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323754    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323757    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323769    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323775    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323777    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323780    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323784    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323787    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323790    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323792    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323796    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323800    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323804    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323807    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323826    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323829    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 13:13:35.888343 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323832    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323834    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323837    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323840    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323843    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323847    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323849    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323852    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323855    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323857    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323860    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323863    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323865    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323868    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323870    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323873    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323876    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323878    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323881    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 13:13:35.888881 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323885    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323887    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323890    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323892    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323895    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323898    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323901    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323904    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323906    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323910    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323912    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323915    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323918    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323920    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323923    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323926    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323928    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323931    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323934    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323936    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 13:13:35.889403 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323939    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323943    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323946    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323949    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323952    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323955    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323958    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323960    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323963    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323965    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323968    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323971    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323974    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323977    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323980    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323982    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323985    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323988    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323990    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 13:13:35.889938 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323993    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323996    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.323999    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324002    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324005    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324007    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324010    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324013    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324015    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324019    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324023    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324025    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324029    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324031    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324034    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324037    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324040    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324042    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 13:13:35.890457 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:35.324045    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.324050    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.324161    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.326651    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.328232    2577 server.go:1019] "Starting client certificate rotation"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.328335    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.328371    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.331111    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.334668    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.342228    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.346555    2577 log.go:25] "Validated CRI v1 image API"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.347364    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 13:13:35.890984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.353291    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b256d7bd-80f5-481f-8b47-c4b0f910f8ac:/dev/nvme0n1p3 e405dcf8-7fee-4fe9-b97d-98a66a6f85a9:/dev/nvme0n1p4]
Jul 07 13:13:35.891363 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.353311    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 13:13:35.891363 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.357526    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.358486    2577 manager.go:216] Machine: {Timestamp:2026-07-07 13:13:35.357142268 +0000 UTC m=+0.374298463 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106605 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2626d4ed99a0a89d6a15c94bd3620d SystemUUID:ec2626d4-ed99-a0a8-9d6a-15c94bd3620d BootID:50dc515a-0b0c-4d7c-85d2-6f36b71a13bd Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:95:a3:d6:78:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:95:a3:d6:78:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:0d:00:52:8c:ce Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.358575    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.358641    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.359782    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.359805    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-146.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.360653    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.360665    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.360672    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.361469    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.363396    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.368087    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.368192    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.368218    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.368227    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.368241    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.368251    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.368259    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 13:13:35.891528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.369553    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.369583    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.372340    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.375069    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.375095    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376458    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376471    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376478    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376484    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376493    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376499    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376507    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376512    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376519    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376525    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376533    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.376538    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.377242    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.377251    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.380192    2577 server.go:1268] "Started kubelet"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.380301    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.380259    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.380371    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.381621    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.382636    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.384840    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.384859    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.384905    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-146.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.386265    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.392012    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.392028    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.392160    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.392753    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.392755    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.392786    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.392944    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.392954    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.393158    2577 factory.go:55] Registering systemd factory
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.393169    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.393384    2577 factory.go:153] Registering CRI-O factory
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.393400    2577 factory.go:223] Registration of the crio container factory successfully
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.393445    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.393471    2577 factory.go:103] Registering Raw factory
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.393485    2577 manager.go:1196] Started watching for new ooms in manager
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.394393    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q9k76"
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.395383    2577 manager.go:349] Starting recovery of all containers
Jul 07 13:13:35.892497 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.396375    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.396648    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.396445    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c0042664811585  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-07 13:13:35.380166021 +0000 UTC m=+0.397322195,LastTimestamp:2026-07-07 13:13:35.380166021 +0000 UTC m=+0.397322195,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.403878    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q9k76"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.405955    2577 manager.go:354] Recovery completed
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.409675    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.411298    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.411326    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.411336    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.411857    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.411869    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.411876    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.411885    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.412487    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.412509    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.412519    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.413713    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c00426665c5707  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-07 13:13:35.411312391 +0000 UTC m=+0.428468567,LastTimestamp:2026-07-07 13:13:35.411312391 +0000 UTC m=+0.428468567,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.413978    2577 policy_none.go:49] "None policy: Start"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.413995    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.414004    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.415975    2577 policy_none.go:47] "Start"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.475312    2577 manager.go:337] "Starting Device Plugin manager"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.475463    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.475476    2577 server.go:87] "Starting device plugin registration server"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.475658    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.475670    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.475806    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.475887    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.475896    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.476965    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.476999    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.506604    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.507710    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.507744    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.507772    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 13:13:35.893749 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.507807    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.510401    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.576475    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.577241    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.577267    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.577284    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.577308    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.600168    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.600182    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-146.ec2.internal\": node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.608848    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"]
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.609357    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.610057    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.610078    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.610088    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614205    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614316    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614342    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614866    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614872    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614892    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614895    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614902    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.614908    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.617189    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.617214    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.618196    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.618220    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.618237    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.643565    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.655357    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-146.ec2.internal\" not found" node="ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.659623    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-146.ec2.internal\" not found" node="ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.694914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/501e629b6abaa0cc412055f9dc338d00-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"501e629b6abaa0cc412055f9dc338d00\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.694936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501e629b6abaa0cc412055f9dc338d00-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"501e629b6abaa0cc412055f9dc338d00\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.127429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.694953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0896f86a31d2ff3cbf0414b475c0fa9a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-146.ec2.internal\" (UID: \"0896f86a31d2ff3cbf0414b475c0fa9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.744615    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.795684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0896f86a31d2ff3cbf0414b475c0fa9a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-146.ec2.internal\" (UID: \"0896f86a31d2ff3cbf0414b475c0fa9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.795709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/501e629b6abaa0cc412055f9dc338d00-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"501e629b6abaa0cc412055f9dc338d00\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.795726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501e629b6abaa0cc412055f9dc338d00-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"501e629b6abaa0cc412055f9dc338d00\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.795772    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501e629b6abaa0cc412055f9dc338d00-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"501e629b6abaa0cc412055f9dc338d00\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.795784    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/501e629b6abaa0cc412055f9dc338d00-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"501e629b6abaa0cc412055f9dc338d00\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.795784    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0896f86a31d2ff3cbf0414b475c0fa9a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-146.ec2.internal\" (UID: \"0896f86a31d2ff3cbf0414b475c0fa9a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.844774    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:35.945402    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.958488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:35.963467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.129039 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:36.046441    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.364376 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:36.146937    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.364376 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:36.247387    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.364376 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.327902    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 13:13:36.364376 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.328016    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 13:13:36.364376 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:36.348173    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.388578    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.393944    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.393958    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.406072    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.406822    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 13:08:35 +0000 UTC" deadline="2027-12-14 19:28:14.825617105 +0000 UTC"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.406843    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12606h14m38.418776393s"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.407525    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.411369    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.417405    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.430163    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.434827    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6rndl"
Jul 07 13:13:36.560962 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.443546    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6rndl"
Jul 07 13:13:36.563585 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:36.563258    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501e629b6abaa0cc412055f9dc338d00.slice/crio-7d0bcd55467a3d1437446abf54682ec02360b053ee68f3bbef462f3d06d45bc8 WatchSource:0}: Error finding container 7d0bcd55467a3d1437446abf54682ec02360b053ee68f3bbef462f3d06d45bc8: Status 404 returned error can't find the container with id 7d0bcd55467a3d1437446abf54682ec02360b053ee68f3bbef462f3d06d45bc8
Jul 07 13:13:36.836062 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.569893    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:13:36.836062 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:36.577188    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0896f86a31d2ff3cbf0414b475c0fa9a.slice/crio-8eb226dfeb7797e30348479890719713ad6166a2964b8f968cb54e1912e1ddbc WatchSource:0}: Error finding container 8eb226dfeb7797e30348479890719713ad6166a2964b8f968cb54e1912e1ddbc: Status 404 returned error can't find the container with id 8eb226dfeb7797e30348479890719713ad6166a2964b8f968cb54e1912e1ddbc
Jul 07 13:13:37.081838 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:36.904578    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:13:37.378153 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.369438    2577 apiserver.go:52] "Watching apiserver"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.404898    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.405496    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-gg9gk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal","openshift-multus/multus-7w9c6","openshift-image-registry/node-ca-zm2x9","openshift-multus/multus-additional-cni-plugins-jjlc2","openshift-multus/network-metrics-daemon-ndrqj","openshift-network-diagnostics/network-check-target-d2bxr","openshift-network-operator/iptables-alerter-ch8kf","openshift-ovn-kubernetes/ovnkube-node-qpsjj","kube-system/global-pull-secret-syncer-889k8","kube-system/konnectivity-agent-rbcrr","kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx","openshift-cluster-node-tuning-operator/tuned-c7x7s"]
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.408554    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.408632    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.410809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.413019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.413091    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.415341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.418093    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.427257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.429889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.434642    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.434675    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.434706    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jbx84\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.436022    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.437461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.438553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.438651    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.439930    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lqnpt\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.440490    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.440493    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.440836    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.441231    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m5j59\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.441525    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.441791    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.441880    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.442731    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.446580    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:08:36 +0000 UTC" deadline="2028-01-21 09:47:02.995654983 +0000 UTC"
Jul 07 13:13:37.485234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.446607    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13508h33m25.54905174s"
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.446680    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.446739    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.446958    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mlhzc\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.447136    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.447246    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.447268    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.447310    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ct99h\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.447140    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.450323    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.450551    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.450782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.450936    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.451077    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-drj2w\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.451095    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.451466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.451623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.451725    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.451969    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.452308    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.452551    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.452580    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ms4lq\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.452813    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.453087    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lpvts\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.454383    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.455880    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 13:13:37.486840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.456176    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m5cx6\""
Jul 07 13:13:37.495190 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.494116    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/040975fe-ab93-4030-9fe7-1fbf8b92a86a-kubelet-config\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-slash\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8036e813-c733-4d59-9fa0-64170a420795-etc-tuned\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-host\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/040975fe-ab93-4030-9fe7-1fbf8b92a86a-dbus\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503391    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-os-release\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-268wx\" (UniqueName: \"kubernetes.io/projected/520d0a12-2834-46f1-b42a-6f4a23c9037f-kube-api-access-268wx\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-kubelet\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-socket-dir-parent\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-conf-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.504866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503606    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-sys-fs\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjctd\" (UniqueName: \"kubernetes.io/projected/62f5f556-db8b-4d78-a246-7e68a809d40f-kube-api-access-mjctd\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-daemon-config\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-run-netns\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-env-overrides\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-registration-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8036e813-c733-4d59-9fa0-64170a420795-tmp\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-cni-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-k8s-cni-cncf-io\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-cni-netd\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-socket-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-modprobe-d\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-os-release\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb3ea164-333c-4158-ad32-37dccca24c44-agent-certs\") pod \"konnectivity-agent-rbcrr\" (UID: \"fb3ea164-333c-4158-ad32-37dccca24c44\") " pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.505504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.503993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/520d0a12-2834-46f1-b42a-6f4a23c9037f-hosts-file\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-run-ovn-kubernetes\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysconfig\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-netns\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khxpf\" (UniqueName: \"kubernetes.io/projected/46d55b76-0090-47ac-abab-d302e95ed7c1-kube-api-access-khxpf\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520d0a12-2834-46f1-b42a-6f4a23c9037f-tmp-dir\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-device-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-var-lib-kubelet\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-host\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-cni-bin\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-systemd\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw92j\" (UniqueName: \"kubernetes.io/projected/9311d236-5f18-4f5d-bbea-a777f321d79a-kube-api-access-dw92j\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-ovn\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-ovnkube-config\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.570868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9311d236-5f18-4f5d-bbea-a777f321d79a-cni-binary-copy\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46d55b76-0090-47ac-abab-d302e95ed7c1-iptables-alerter-script\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwrqr\" (UniqueName: \"kubernetes.io/projected/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-kube-api-access-zwrqr\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-log-socket\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-system-cni-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-system-cni-dir\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rxjk\" (UniqueName: \"kubernetes.io/projected/8036e813-c733-4d59-9fa0-64170a420795-kube-api-access-2rxjk\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-systemd-units\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-var-lib-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccd04ae1-3fef-4266-9822-7da10240a092-ovn-node-metrics-cert\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvr5b\" (UniqueName: \"kubernetes.io/projected/ccd04ae1-3fef-4266-9822-7da10240a092-kube-api-access-hvr5b\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-cni-multus\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-multus-certs\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-etc-kubernetes\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.572192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46d55b76-0090-47ac-abab-d302e95ed7c1-host-slash\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb9q2\" (UniqueName: \"kubernetes.io/projected/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-kube-api-access-fb9q2\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysctl-d\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-sys\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-etc-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-kubernetes\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysctl-conf\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-cni-bin\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-hostroot\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb3ea164-333c-4158-ad32-37dccca24c44-konnectivity-ca\") pod \"konnectivity-agent-rbcrr\" (UID: \"fb3ea164-333c-4158-ad32-37dccca24c44\") " pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cnibin\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-systemd\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.504994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-ovnkube-script-lib\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.505016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-run\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.505038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d27pj\" (UniqueName: \"kubernetes.io/projected/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-kube-api-access-d27pj\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.505061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-cnibin\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.505084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-kubelet\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.573769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.505107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-node-log\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.574720 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.505127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-lib-modules\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.574720 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.505149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-serviceca\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.574720 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.516753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal" event={"ID":"0896f86a31d2ff3cbf0414b475c0fa9a","Type":"ContainerStarted","Data":"8eb226dfeb7797e30348479890719713ad6166a2964b8f968cb54e1912e1ddbc"}
Jul 07 13:13:37.574720 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.518665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" event={"ID":"501e629b6abaa0cc412055f9dc338d00","Type":"ContainerStarted","Data":"7d0bcd55467a3d1437446abf54682ec02360b053ee68f3bbef462f3d06d45bc8"}
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-multus-certs\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-etc-kubernetes\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46d55b76-0090-47ac-abab-d302e95ed7c1-host-slash\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb9q2\" (UniqueName: \"kubernetes.io/projected/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-kube-api-access-fb9q2\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysctl-d\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-sys\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-etc-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-kubernetes\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysctl-conf\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-cni-bin\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-hostroot\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb3ea164-333c-4158-ad32-37dccca24c44-konnectivity-ca\") pod \"konnectivity-agent-rbcrr\" (UID: \"fb3ea164-333c-4158-ad32-37dccca24c44\") " pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cnibin\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-systemd\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-ovnkube-script-lib\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-run\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d27pj\" (UniqueName: \"kubernetes.io/projected/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-kube-api-access-d27pj\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.645345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-cnibin\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-kubelet\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-node-log\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-lib-modules\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-serviceca\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/040975fe-ab93-4030-9fe7-1fbf8b92a86a-kubelet-config\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-slash\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8036e813-c733-4d59-9fa0-64170a420795-etc-tuned\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-host\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/040975fe-ab93-4030-9fe7-1fbf8b92a86a-dbus\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-os-release\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-268wx\" (UniqueName: \"kubernetes.io/projected/520d0a12-2834-46f1-b42a-6f4a23c9037f-kube-api-access-268wx\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-kubelet\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:37.648422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-socket-dir-parent\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-conf-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-sys-fs\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjctd\" (UniqueName: \"kubernetes.io/projected/62f5f556-db8b-4d78-a246-7e68a809d40f-kube-api-access-mjctd\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-daemon-config\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-run-netns\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-env-overrides\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-registration-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8036e813-c733-4d59-9fa0-64170a420795-tmp\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-cni-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-k8s-cni-cncf-io\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-cni-netd\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-socket-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-modprobe-d\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.649607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-os-release\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb3ea164-333c-4158-ad32-37dccca24c44-agent-certs\") pod \"konnectivity-agent-rbcrr\" (UID: \"fb3ea164-333c-4158-ad32-37dccca24c44\") " pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/520d0a12-2834-46f1-b42a-6f4a23c9037f-hosts-file\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-run-ovn-kubernetes\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysconfig\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-netns\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khxpf\" (UniqueName: \"kubernetes.io/projected/46d55b76-0090-47ac-abab-d302e95ed7c1-kube-api-access-khxpf\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520d0a12-2834-46f1-b42a-6f4a23c9037f-tmp-dir\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-device-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-var-lib-kubelet\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-host\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-cni-bin\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607921    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/040975fe-ab93-4030-9fe7-1fbf8b92a86a-dbus\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-systemd\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605907    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-etc-kubernetes\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw92j\" (UniqueName: \"kubernetes.io/projected/9311d236-5f18-4f5d-bbea-a777f321d79a-kube-api-access-dw92j\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.607999    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-sys\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.652616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605944    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-multus-certs\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-ovn\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608044    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-etc-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-ovnkube-config\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.605985    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46d55b76-0090-47ac-abab-d302e95ed7c1-host-slash\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608102    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-kubernetes\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.606094    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysctl-d\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9311d236-5f18-4f5d-bbea-a777f321d79a-cni-binary-copy\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46d55b76-0090-47ac-abab-d302e95ed7c1-iptables-alerter-script\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwrqr\" (UniqueName: \"kubernetes.io/projected/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-kube-api-access-zwrqr\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-log-socket\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-system-cni-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-system-cni-dir\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rxjk\" (UniqueName: \"kubernetes.io/projected/8036e813-c733-4d59-9fa0-64170a420795-kube-api-access-2rxjk\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-systemd-units\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.654051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-var-lib-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccd04ae1-3fef-4266-9822-7da10240a092-ovn-node-metrics-cert\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvr5b\" (UniqueName: \"kubernetes.io/projected/ccd04ae1-3fef-4266-9822-7da10240a092-kube-api-access-hvr5b\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-cni-multus\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608595    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-cni-multus\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608684    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9311d236-5f18-4f5d-bbea-a777f321d79a-cni-binary-copy\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608722    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysctl-conf\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608765    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-cni-bin\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608773    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-os-release\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608818    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-hostroot\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.608825    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cnibin\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.609391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb3ea164-333c-4158-ad32-37dccca24c44-konnectivity-ca\") pod \"konnectivity-agent-rbcrr\" (UID: \"fb3ea164-333c-4158-ad32-37dccca24c44\") " pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.609447    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.609454    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-systemd\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.609494    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-kubelet\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.609518    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-socket-dir-parent\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.609561    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-conf-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.658143 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.609582    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.609662    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs podName:62f5f556-db8b-4d78-a246-7e68a809d40f nodeName:}" failed. No retries permitted until 2026-07-07 13:13:38.10964151 +0000 UTC m=+3.126797687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs") pod "network-metrics-daemon-ndrqj" (UID: "62f5f556-db8b-4d78-a246-7e68a809d40f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610034    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610114    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-sys-fs\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610201    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-ovnkube-script-lib\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610263    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-run\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610531    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-cnibin\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610585    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-var-lib-kubelet\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610630    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-node-log\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610733    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-lib-modules\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.610782    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-daemon-config\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.610865    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.610937    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret podName:040975fe-ab93-4030-9fe7-1fbf8b92a86a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:38.110917229 +0000 UTC m=+3.128073394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret") pod "global-pull-secret-syncer-889k8" (UID: "040975fe-ab93-4030-9fe7-1fbf8b92a86a") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611078    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611130    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-run-netns\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611155    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-serviceca\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611218    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/040975fe-ab93-4030-9fe7-1fbf8b92a86a-kubelet-config\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:37.659533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611262    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-slash\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611306    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611514    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-env-overrides\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611588    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-registration-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611608    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.611903    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520d0a12-2834-46f1-b42a-6f4a23c9037f-tmp-dir\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.612170    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.612182    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-device-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.612246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-var-lib-kubelet\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.612297    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-host\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.612339    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-cni-bin\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.612399    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-systemd\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.612660    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-run-ovn\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.613140    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ccd04ae1-3fef-4266-9822-7da10240a092-ovnkube-config\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.613206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.613432    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-host\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.614144    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-systemd-units\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.614202    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-var-lib-openvswitch\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.663569 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.614614    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-os-release\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.614680    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-multus-cni-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615014    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46d55b76-0090-47ac-abab-d302e95ed7c1-iptables-alerter-script\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615209    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-k8s-cni-cncf-io\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615265    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-cni-netd\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615365    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-log-socket\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.609391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-modprobe-d\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615462    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615685    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-system-cni-dir\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615736    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccd04ae1-3fef-4266-9822-7da10240a092-host-run-ovn-kubernetes\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.616029    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8036e813-c733-4d59-9fa0-64170a420795-tmp\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.616104    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8036e813-c733-4d59-9fa0-64170a420795-etc-sysconfig\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.616133    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-system-cni-dir\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.616146    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9311d236-5f18-4f5d-bbea-a777f321d79a-host-run-netns\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.615372    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-socket-dir\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.616211    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/520d0a12-2834-46f1-b42a-6f4a23c9037f-hosts-file\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.618737    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8036e813-c733-4d59-9fa0-64170a420795-etc-tuned\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.664756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.619031    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccd04ae1-3fef-4266-9822-7da10240a092-ovn-node-metrics-cert\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.670937 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.619434    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb3ea164-333c-4158-ad32-37dccca24c44-agent-certs\") pod \"konnectivity-agent-rbcrr\" (UID: \"fb3ea164-333c-4158-ad32-37dccca24c44\") " pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.696635    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.696669    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.696684    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lf2tt for pod openshift-network-diagnostics/network-check-target-d2bxr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:37.696765    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt podName:fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:38.196734026 +0000 UTC m=+3.213890189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lf2tt" (UniqueName: "kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt") pod "network-check-target-d2bxr" (UID: "fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.696947    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.699195    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb9q2\" (UniqueName: \"kubernetes.io/projected/ddd7b144-b67d-4a1b-8d1f-33bee3b6390b-kube-api-access-fb9q2\") pod \"aws-ebs-csi-driver-node-w8jsx\" (UID: \"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.699721    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvr5b\" (UniqueName: \"kubernetes.io/projected/ccd04ae1-3fef-4266-9822-7da10240a092-kube-api-access-hvr5b\") pod \"ovnkube-node-qpsjj\" (UID: \"ccd04ae1-3fef-4266-9822-7da10240a092\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.700313    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjctd\" (UniqueName: \"kubernetes.io/projected/62f5f556-db8b-4d78-a246-7e68a809d40f-kube-api-access-mjctd\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.700654    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rxjk\" (UniqueName: \"kubernetes.io/projected/8036e813-c733-4d59-9fa0-64170a420795-kube-api-access-2rxjk\") pod \"tuned-c7x7s\" (UID: \"8036e813-c733-4d59-9fa0-64170a420795\") " pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.701436    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwrqr\" (UniqueName: \"kubernetes.io/projected/23e93a3f-c550-4fbb-9ef5-80fd6a249da6-kube-api-access-zwrqr\") pod \"multus-additional-cni-plugins-jjlc2\" (UID: \"23e93a3f-c550-4fbb-9ef5-80fd6a249da6\") " pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.702048    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw92j\" (UniqueName: \"kubernetes.io/projected/9311d236-5f18-4f5d-bbea-a777f321d79a-kube-api-access-dw92j\") pod \"multus-7w9c6\" (UID: \"9311d236-5f18-4f5d-bbea-a777f321d79a\") " pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.702392    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khxpf\" (UniqueName: \"kubernetes.io/projected/46d55b76-0090-47ac-abab-d302e95ed7c1-kube-api-access-khxpf\") pod \"iptables-alerter-ch8kf\" (UID: \"46d55b76-0090-47ac-abab-d302e95ed7c1\") " pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.703437    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d27pj\" (UniqueName: \"kubernetes.io/projected/0e132ef7-c007-4cec-b2f4-83ebbc940c9e-kube-api-access-d27pj\") pod \"node-ca-zm2x9\" (UID: \"0e132ef7-c007-4cec-b2f4-83ebbc940c9e\") " pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.704554    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-268wx\" (UniqueName: \"kubernetes.io/projected/520d0a12-2834-46f1-b42a-6f4a23c9037f-kube-api-access-268wx\") pod \"node-resolver-gg9gk\" (UID: \"520d0a12-2834-46f1-b42a-6f4a23c9037f\") " pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.728287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ch8kf"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.752356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zm2x9"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.760407    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jjlc2"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.766381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7w9c6"
Jul 07 13:13:37.773769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.773525    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gg9gk"
Jul 07 13:13:37.852351 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.852330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:13:37.852465 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.852390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:37.853405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.853155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx"
Jul 07 13:13:37.853405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:37.853220    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c7x7s"
Jul 07 13:13:38.155022 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:38.154944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:38.155022 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:38.154994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:38.155198 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.155120    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:38.155198 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.155186    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs podName:62f5f556-db8b-4d78-a246-7e68a809d40f nodeName:}" failed. No retries permitted until 2026-07-07 13:13:39.155171889 +0000 UTC m=+4.172328055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs") pod "network-metrics-daemon-ndrqj" (UID: "62f5f556-db8b-4d78-a246-7e68a809d40f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:38.155297 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.155120    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:38.155356 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.155309    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret podName:040975fe-ab93-4030-9fe7-1fbf8b92a86a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:39.155284867 +0000 UTC m=+4.172441032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret") pod "global-pull-secret-syncer-889k8" (UID: "040975fe-ab93-4030-9fe7-1fbf8b92a86a") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:38.256000 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:38.255971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:38.256145 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.256108    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:13:38.256145 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.256131    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:13:38.256145 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.256145    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lf2tt for pod openshift-network-diagnostics/network-check-target-d2bxr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:38.256283 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:38.256194    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt podName:fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:39.256180778 +0000 UTC m=+4.273336941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lf2tt" (UniqueName: "kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt") pod "network-check-target-d2bxr" (UID: "fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:38.447622 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:38.447529    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:08:36 +0000 UTC" deadline="2028-03-29 09:08:05.409713354 +0000 UTC"
Jul 07 13:13:38.447622 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:38.447581    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15139h54m26.962135943s"
Jul 07 13:13:38.815773 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:38.815743    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod520d0a12_2834_46f1_b42a_6f4a23c9037f.slice/crio-8d6b32b5454a110af0606ca9e95900b0fb011bf097880a3390a139feb84eb833 WatchSource:0}: Error finding container 8d6b32b5454a110af0606ca9e95900b0fb011bf097880a3390a139feb84eb833: Status 404 returned error can't find the container with id 8d6b32b5454a110af0606ca9e95900b0fb011bf097880a3390a139feb84eb833
Jul 07 13:13:38.818432 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:38.818169    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3ea164_333c_4158_ad32_37dccca24c44.slice/crio-3ce48958fffe589ac487e78eea990164307da2dc7eb6f33b4dd80f8ebd7e8750 WatchSource:0}: Error finding container 3ce48958fffe589ac487e78eea990164307da2dc7eb6f33b4dd80f8ebd7e8750: Status 404 returned error can't find the container with id 3ce48958fffe589ac487e78eea990164307da2dc7eb6f33b4dd80f8ebd7e8750
Jul 07 13:13:38.821446 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:38.821355    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd04ae1_3fef_4266_9822_7da10240a092.slice/crio-5edfe47b82533f3abe806a905b2da582d0930961aa36b4ee1c692534fb31ac49 WatchSource:0}: Error finding container 5edfe47b82533f3abe806a905b2da582d0930961aa36b4ee1c692534fb31ac49: Status 404 returned error can't find the container with id 5edfe47b82533f3abe806a905b2da582d0930961aa36b4ee1c692534fb31ac49
Jul 07 13:13:38.822174 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:38.822147    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9311d236_5f18_4f5d_bbea_a777f321d79a.slice/crio-b8954d9671cc3f40974eecbf04d2e3acad4c3e92e4a11116198e50cb6a739fdc WatchSource:0}: Error finding container b8954d9671cc3f40974eecbf04d2e3acad4c3e92e4a11116198e50cb6a739fdc: Status 404 returned error can't find the container with id b8954d9671cc3f40974eecbf04d2e3acad4c3e92e4a11116198e50cb6a739fdc
Jul 07 13:13:38.823209 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:38.823166    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e132ef7_c007_4cec_b2f4_83ebbc940c9e.slice/crio-c4c4b1b26d1b0cd2268834ee47a9a374dd89dd0b8a7e96b0475800d8910f67ab WatchSource:0}: Error finding container c4c4b1b26d1b0cd2268834ee47a9a374dd89dd0b8a7e96b0475800d8910f67ab: Status 404 returned error can't find the container with id c4c4b1b26d1b0cd2268834ee47a9a374dd89dd0b8a7e96b0475800d8910f67ab
Jul 07 13:13:38.825214 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:38.824237    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd7b144_b67d_4a1b_8d1f_33bee3b6390b.slice/crio-6d7a2bdd67c1c5187cfb81f9a7005334d71dce82ff009667d9711fde462f1125 WatchSource:0}: Error finding container 6d7a2bdd67c1c5187cfb81f9a7005334d71dce82ff009667d9711fde462f1125: Status 404 returned error can't find the container with id 6d7a2bdd67c1c5187cfb81f9a7005334d71dce82ff009667d9711fde462f1125
Jul 07 13:13:38.825662 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:13:38.825517    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8036e813_c733_4d59_9fa0_64170a420795.slice/crio-07731b56bb12946bf89384488a0aa4c91fb7a0e7c5393ef35fe943e3137da529 WatchSource:0}: Error finding container 07731b56bb12946bf89384488a0aa4c91fb7a0e7c5393ef35fe943e3137da529: Status 404 returned error can't find the container with id 07731b56bb12946bf89384488a0aa4c91fb7a0e7c5393ef35fe943e3137da529
Jul 07 13:13:39.161801 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.161654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:39.161918 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.161806    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:39.161918 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.161826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:39.161918 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.161875    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs podName:62f5f556-db8b-4d78-a246-7e68a809d40f nodeName:}" failed. No retries permitted until 2026-07-07 13:13:41.161859053 +0000 UTC m=+6.179015216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs") pod "network-metrics-daemon-ndrqj" (UID: "62f5f556-db8b-4d78-a246-7e68a809d40f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:39.162071 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.161933    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:39.162071 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.161991    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret podName:040975fe-ab93-4030-9fe7-1fbf8b92a86a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:41.161973973 +0000 UTC m=+6.179130152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret") pod "global-pull-secret-syncer-889k8" (UID: "040975fe-ab93-4030-9fe7-1fbf8b92a86a") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:39.262884 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.262859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:39.262984 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.262969    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:13:39.262984 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.262983    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:13:39.263077 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.262992    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lf2tt for pod openshift-network-diagnostics/network-check-target-d2bxr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:39.263077 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.263035    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt podName:fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:41.263023249 +0000 UTC m=+6.280179412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lf2tt" (UniqueName: "kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt") pod "network-check-target-d2bxr" (UID: "fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:39.513156 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.512899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:39.513156 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.513017    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:39.513447 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.513419    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:39.513558 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.513535    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:39.513736 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.513607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:39.513736 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:39.513680    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:39.528229 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.528198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gg9gk" event={"ID":"520d0a12-2834-46f1-b42a-6f4a23c9037f","Type":"ContainerStarted","Data":"8d6b32b5454a110af0606ca9e95900b0fb011bf097880a3390a139feb84eb833"}
Jul 07 13:13:39.531774 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.531744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal" event={"ID":"0896f86a31d2ff3cbf0414b475c0fa9a","Type":"ContainerStarted","Data":"847b699c476c35c010e486ab9181ff1bfeb12bfebbfcf7f5b6d717c2b32bc7c6"}
Jul 07 13:13:39.534337 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.534303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerStarted","Data":"4fe810360d2adb4ed7e5e9dacb478077626466ae95bb033bc0ad9c218a3f696d"}
Jul 07 13:13:39.537221 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.537151    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zm2x9" event={"ID":"0e132ef7-c007-4cec-b2f4-83ebbc940c9e","Type":"ContainerStarted","Data":"c4c4b1b26d1b0cd2268834ee47a9a374dd89dd0b8a7e96b0475800d8910f67ab"}
Jul 07 13:13:39.540852 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.540780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7w9c6" event={"ID":"9311d236-5f18-4f5d-bbea-a777f321d79a","Type":"ContainerStarted","Data":"b8954d9671cc3f40974eecbf04d2e3acad4c3e92e4a11116198e50cb6a739fdc"}
Jul 07 13:13:39.544494 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.544312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"5edfe47b82533f3abe806a905b2da582d0930961aa36b4ee1c692534fb31ac49"}
Jul 07 13:13:39.547328 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.546699    2577 generic.go:356] "Generic (PLEG): container finished" podID="501e629b6abaa0cc412055f9dc338d00" containerID="4328ec47e27b69068b43e5909fb7fa7af7719af372b486a40d144b453ff8dd75" exitCode=0
Jul 07 13:13:39.547328 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.546770    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" event={"ID":"501e629b6abaa0cc412055f9dc338d00","Type":"ContainerDied","Data":"4328ec47e27b69068b43e5909fb7fa7af7719af372b486a40d144b453ff8dd75"}
Jul 07 13:13:39.550041 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.550014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c7x7s" event={"ID":"8036e813-c733-4d59-9fa0-64170a420795","Type":"ContainerStarted","Data":"07731b56bb12946bf89384488a0aa4c91fb7a0e7c5393ef35fe943e3137da529"}
Jul 07 13:13:39.553106 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.553079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ch8kf" event={"ID":"46d55b76-0090-47ac-abab-d302e95ed7c1","Type":"ContainerStarted","Data":"af8b9c45b054dc126220b55741bcb86a76e7306b03b7a007c216c719cb8c9b98"}
Jul 07 13:13:39.556402 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.556375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx" event={"ID":"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b","Type":"ContainerStarted","Data":"6d7a2bdd67c1c5187cfb81f9a7005334d71dce82ff009667d9711fde462f1125"}
Jul 07 13:13:39.560303 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.560277    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbcrr" event={"ID":"fb3ea164-333c-4158-ad32-37dccca24c44","Type":"ContainerStarted","Data":"3ce48958fffe589ac487e78eea990164307da2dc7eb6f33b4dd80f8ebd7e8750"}
Jul 07 13:13:39.575753 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:39.575694    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal" podStartSLOduration=3.575675697 podStartE2EDuration="3.575675697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:39.554622432 +0000 UTC m=+4.571778633" watchObservedRunningTime="2026-07-07 13:13:39.575675697 +0000 UTC m=+4.592831886"
Jul 07 13:13:40.578035 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:40.577956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" event={"ID":"501e629b6abaa0cc412055f9dc338d00","Type":"ContainerStarted","Data":"d0d0a19132c2b29455398485597240d0e46c8fa3596937f789959698e5ebe5c3"}
Jul 07 13:13:41.179635 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:41.179582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:41.179843 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:41.179671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:41.180314 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.179915    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:41.180314 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.179978    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret podName:040975fe-ab93-4030-9fe7-1fbf8b92a86a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:45.179959725 +0000 UTC m=+10.197115892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret") pod "global-pull-secret-syncer-889k8" (UID: "040975fe-ab93-4030-9fe7-1fbf8b92a86a") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:41.180579 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.179821    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:41.180579 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.180559    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs podName:62f5f556-db8b-4d78-a246-7e68a809d40f nodeName:}" failed. No retries permitted until 2026-07-07 13:13:45.180538375 +0000 UTC m=+10.197694575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs") pod "network-metrics-daemon-ndrqj" (UID: "62f5f556-db8b-4d78-a246-7e68a809d40f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:41.280487 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:41.280218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:41.280487 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.280366    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:13:41.280487 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.280380    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:13:41.280487 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.280392    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lf2tt for pod openshift-network-diagnostics/network-check-target-d2bxr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:41.280487 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.280458    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt podName:fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:45.280432874 +0000 UTC m=+10.297589036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lf2tt" (UniqueName: "kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt") pod "network-check-target-d2bxr" (UID: "fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:41.510079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:41.509935    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:41.510079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:41.509951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:41.510281 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.510079    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:41.510281 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.510177    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:41.510581 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:41.510432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:41.510581 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:41.510542    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:43.508649 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:43.508187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:43.508649 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:43.508308    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:43.508649 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:43.508374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:43.508649 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:43.508394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:43.508649 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:43.508462    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:43.508649 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:43.508552    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:45.214755 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:45.214723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:45.215207 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:45.214774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:45.215207 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.214900    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:45.215207 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.215001    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret podName:040975fe-ab93-4030-9fe7-1fbf8b92a86a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:53.214971207 +0000 UTC m=+18.232127368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret") pod "global-pull-secret-syncer-889k8" (UID: "040975fe-ab93-4030-9fe7-1fbf8b92a86a") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:45.215207 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.214899    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:45.215207 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.215082    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs podName:62f5f556-db8b-4d78-a246-7e68a809d40f nodeName:}" failed. No retries permitted until 2026-07-07 13:13:53.215059442 +0000 UTC m=+18.232215607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs") pod "network-metrics-daemon-ndrqj" (UID: "62f5f556-db8b-4d78-a246-7e68a809d40f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:45.315661 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:45.315037    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:45.315661 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.315232    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:13:45.315661 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.315252    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:13:45.315661 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.315263    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lf2tt for pod openshift-network-diagnostics/network-check-target-d2bxr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:45.315661 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.315316    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt podName:fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:53.315299271 +0000 UTC m=+18.332455435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lf2tt" (UniqueName: "kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt") pod "network-check-target-d2bxr" (UID: "fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:45.509344 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:45.509122    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:45.509344 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.509218    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:45.509344 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:45.509267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:45.509344 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.509342    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:45.509795 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:45.509658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:45.509795 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:45.509755    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:47.508599 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:47.508566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:47.509132 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:47.508572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:47.509132 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:47.508687    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:47.509132 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:47.508576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:47.509132 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:47.508792    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:47.509132 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:47.508858    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:49.508262 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:49.508183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:49.508654 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:49.508196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:49.508654 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:49.508301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:49.508654 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:49.508302    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:49.508654 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:49.508386    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:49.508654 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:49.508459    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:50.606154 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.605939    2577 generic.go:356] "Generic (PLEG): container finished" podID="23e93a3f-c550-4fbb-9ef5-80fd6a249da6" containerID="bcb5e85a096f4302e9751dc873044559f0c2cca7f38b848c02101fcf8383db6f" exitCode=0
Jul 07 13:13:50.606596 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.605991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerDied","Data":"bcb5e85a096f4302e9751dc873044559f0c2cca7f38b848c02101fcf8383db6f"}
Jul 07 13:13:50.609237 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.609214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zm2x9" event={"ID":"0e132ef7-c007-4cec-b2f4-83ebbc940c9e","Type":"ContainerStarted","Data":"3945954c3f97460fe74d71c2a4ea9ced03faf530b235313d51c463e343744689"}
Jul 07 13:13:50.610968 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.610943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c7x7s" event={"ID":"8036e813-c733-4d59-9fa0-64170a420795","Type":"ContainerStarted","Data":"b82e62d054ecd6a180d2ff07322ebfa1d5780841b9b25a3e2d8483214fba64a9"}
Jul 07 13:13:50.614516 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.614489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx" event={"ID":"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b","Type":"ContainerStarted","Data":"729f9f8db57cd45bd17523bbbd53fc779ecfc49df8108d5b895d6dab64ee982b"}
Jul 07 13:13:50.616306 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.616283    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbcrr" event={"ID":"fb3ea164-333c-4158-ad32-37dccca24c44","Type":"ContainerStarted","Data":"3e039fd48ba13b82584d39999227a15aaee052a2972266924c10a4bd1e92f748"}
Jul 07 13:13:50.618136 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.618113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gg9gk" event={"ID":"520d0a12-2834-46f1-b42a-6f4a23c9037f","Type":"ContainerStarted","Data":"82edd824fed0bca5dfdc800ded9fc73a062521ff1f517fa37c4401c8b4fc1cf4"}
Jul 07 13:13:50.632664 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.632620    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" podStartSLOduration=14.632604749 podStartE2EDuration="14.632604749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:40.60042129 +0000 UTC m=+5.617577479" watchObservedRunningTime="2026-07-07 13:13:50.632604749 +0000 UTC m=+15.649760933"
Jul 07 13:13:50.653412 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.653369    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rbcrr" podStartSLOduration=3.990193929 podStartE2EDuration="14.653351381s" totalImagesPullingTime="10.663157452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:50.651099684 +0000 UTC m=+15.668255870" watchObservedRunningTime="2026-07-07 13:13:50.653351381 +0000 UTC m=+15.670507567"
Jul 07 13:13:50.696237 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.696164    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gg9gk" podStartSLOduration=3.9569675650000002 podStartE2EDuration="14.6961509s" totalImagesPullingTime="10.739183335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:50.695422535 +0000 UTC m=+15.712578717" watchObservedRunningTime="2026-07-07 13:13:50.6961509 +0000 UTC m=+15.713307083"
Jul 07 13:13:50.696397 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.696357    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zm2x9" podStartSLOduration=3.964322553 podStartE2EDuration="14.696351074s" totalImagesPullingTime="10.732028521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:50.671220469 +0000 UTC m=+15.688376642" watchObservedRunningTime="2026-07-07 13:13:50.696351074 +0000 UTC m=+15.713507258"
Jul 07 13:13:50.724363 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:50.724308    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c7x7s" podStartSLOduration=3.983510501 podStartE2EDuration="14.724293194s" totalImagesPullingTime="10.740782693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:50.724077316 +0000 UTC m=+15.741233520" watchObservedRunningTime="2026-07-07 13:13:50.724293194 +0000 UTC m=+15.741449377"
Jul 07 13:13:51.252989 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:51.252960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:51.253592 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:51.253568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:51.508652 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:51.508568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:51.508652 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:51.508590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:51.508856 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:51.508697    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:51.508856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:51.508759    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:51.508856 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:51.508838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:51.509021 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:51.508918    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:51.622038 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:51.622003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ch8kf" event={"ID":"46d55b76-0090-47ac-abab-d302e95ed7c1","Type":"ContainerStarted","Data":"508bd788f282afd5ac70c85ac31776e2570272cb6f45b79da69b5d1ecbcc49b4"}
Jul 07 13:13:51.640494 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:51.640446    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ch8kf" podStartSLOduration=5.914043579 podStartE2EDuration="16.64042972s" totalImagesPullingTime="10.726386141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:51.640092019 +0000 UTC m=+16.657248203" watchObservedRunningTime="2026-07-07 13:13:51.64042972 +0000 UTC m=+16.657585903"
Jul 07 13:13:52.623367 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:52.623337    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 13:13:53.272631 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:53.272598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:53.272801 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:53.272641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:53.272801 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.272773    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:53.272801 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.272786    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:53.272967 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.272859    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs podName:62f5f556-db8b-4d78-a246-7e68a809d40f nodeName:}" failed. No retries permitted until 2026-07-07 13:14:09.272838737 +0000 UTC m=+34.289994901 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs") pod "network-metrics-daemon-ndrqj" (UID: "62f5f556-db8b-4d78-a246-7e68a809d40f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:13:53.272967 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.272876    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret podName:040975fe-ab93-4030-9fe7-1fbf8b92a86a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:09.272869582 +0000 UTC m=+34.290025744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret") pod "global-pull-secret-syncer-889k8" (UID: "040975fe-ab93-4030-9fe7-1fbf8b92a86a") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:13:53.373330 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:53.373303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:53.373487 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.373462    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:13:53.373487 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.373485    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:13:53.373586 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.373499    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lf2tt for pod openshift-network-diagnostics/network-check-target-d2bxr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:53.373586 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.373562    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt podName:fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:09.373540739 +0000 UTC m=+34.390696918 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lf2tt" (UniqueName: "kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt") pod "network-check-target-d2bxr" (UID: "fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:13:53.511416 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:53.511386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:53.511583 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:53.511505    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:53.511583 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.511512    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:53.511583 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:53.511521    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:53.511737 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.511629    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:53.511737 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:53.511715    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:55.477522 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:55.477492    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:55.478302 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:55.477691    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 13:13:55.478528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:55.478507    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rbcrr"
Jul 07 13:13:55.509845 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:55.509804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:55.509969 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:55.509919    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:55.509969 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:55.509930    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:55.510085 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:55.510017    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:55.510146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:55.510099    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:55.510206 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:55.510188    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:57.508353 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:57.508322    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:57.508353 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:57.508339    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:57.508353 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:57.508363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:57.509038 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:57.508445    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:57.509038 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:57.508504    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:57.509038 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:57.508585    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:58.614278 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:58.614122    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 13:13:58.637443 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:58.637407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx" event={"ID":"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b","Type":"ContainerStarted","Data":"001d69321b5842cd7353c0b82d8880a4ae3ef9d843becc08a652e44f26124b9d"}
Jul 07 13:13:58.639182 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:58.639155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7w9c6" event={"ID":"9311d236-5f18-4f5d-bbea-a777f321d79a","Type":"ContainerStarted","Data":"31ddfb796e25a7d381530b5c2d0c2ebb6e2829d3552b9ca3264aa7e20c5665a7"}
Jul 07 13:13:58.641745 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:58.641724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"2bda23ad43728bcabdb2fc84b14669aea296cc47b71dd50f9402b5d6f6d3557a"}
Jul 07 13:13:58.662894 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:58.662744    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7w9c6" podStartSLOduration=2.9793234650000002 podStartE2EDuration="22.66272538s" totalImagesPullingTime="19.683401915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:58.662644957 +0000 UTC m=+23.679801141" watchObservedRunningTime="2026-07-07 13:13:58.66272538 +0000 UTC m=+23.679881564"
Jul 07 13:13:59.491202 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.490950    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T13:13:58.614261622Z","UUID":"3141d793-47bc-4983-84d5-2fb1f3cee587","Handler":null,"Name":"","Endpoint":""}
Jul 07 13:13:59.492702 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.492667    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 13:13:59.492702 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.492705    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 13:13:59.508848 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.508825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:13:59.508985 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.508942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:13:59.508985 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:59.508946    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:13:59.509097 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:59.509051    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:13:59.509153 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.509119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:13:59.509232 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:13:59.509204    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:13:59.646840 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.646798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"d2d9a69cd3e1da99f8cc89ad4efff6a1be392c61f3b2c2ab0d75ee851287baa6"}
Jul 07 13:13:59.647202 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.646876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"2fcdeabdd90b56db94be14d8a4cb900b4c4090195f170b944128bf49c2d7c017"}
Jul 07 13:13:59.647202 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.646919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"6a9a3f6dec96e54d9ca75ccb51c17aca26a6110c9e416d5508a037cf5e409e24"}
Jul 07 13:13:59.647202 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.646950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"a750390aa82161b6fc6777a29f175df229e0d72d5b62eb11496a476efdc6b900"}
Jul 07 13:13:59.647202 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:13:59.646982    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"158514008e0b84cb718c07c52284b57e04b30a0cc6978c2ceab1fdda542c1875"}
Jul 07 13:14:00.650650 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:00.650614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx" event={"ID":"ddd7b144-b67d-4a1b-8d1f-33bee3b6390b","Type":"ContainerStarted","Data":"0d60d7d7f4bede8cf4dd465bfead1e14177c8cd896a48c69cb646479720d4719"}
Jul 07 13:14:00.681180 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:00.681128    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8jsx" podStartSLOduration=3.70591087 podStartE2EDuration="24.681112588s" totalImagesPullingTime="20.975201718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:00.680157503 +0000 UTC m=+25.697313708" watchObservedRunningTime="2026-07-07 13:14:00.681112588 +0000 UTC m=+25.698268771"
Jul 07 13:14:01.511544 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:01.511519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:01.511699 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:01.511618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:01.511699 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:01.511621    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:14:01.511699 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:01.511625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:01.511805 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:01.511700    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:14:01.511805 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:01.511759    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:14:01.655802 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:01.655763    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"718353b3fd6269a3b76f47826933df22ed114e99fb03761b72ccb051e66a8734"}
Jul 07 13:14:02.408269 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:02.408227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:02.504158 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:02.504126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:02.580972 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:02.580928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:03.511526 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.511502    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:03.512201 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:03.511604    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:14:03.512201 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.511681    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:03.512201 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:03.511756    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:14:03.512201 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.511792    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:03.512201 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:03.511874    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:14:03.514229 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.514207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:03.592507 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.592467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:03.664347 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.664309    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" event={"ID":"ccd04ae1-3fef-4266-9822-7da10240a092","Type":"ContainerStarted","Data":"f138e17d3fee7cd9c76227cd030ce8f525ee81da371fae5a565c2863dd0accc4"}
Jul 07 13:14:03.664650 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.664628    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:14:03.664739 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.664720    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:14:03.674311 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.674235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:03.681006 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.680981    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:14:03.701525 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.701485    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj" podStartSLOduration=7.976138992 podStartE2EDuration="27.701467156s" totalImagesPullingTime="19.725328164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:03.700768215 +0000 UTC m=+28.717924425" watchObservedRunningTime="2026-07-07 13:14:03.701467156 +0000 UTC m=+28.718623341"
Jul 07 13:14:03.778169 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.778108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:03.882932 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:03.882901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:04.486916 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:04.486780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:04.564155 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:04.564122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:04.667430 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:04.667402    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:14:04.683958 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:04.683935    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:14:04.968197 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:04.968125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:05.189442 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.189412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:05.449705 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.449675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-889k8"]
Jul 07 13:14:05.449844 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.449796    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:05.449913 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:05.449894    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:14:05.452970 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.452944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d2bxr"]
Jul 07 13:14:05.453072 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.453049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:05.453196 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:05.453141    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:14:05.453749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.453726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndrqj"]
Jul 07 13:14:05.453868 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.453854    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:05.453981 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:05.453961    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:14:05.581639 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.581601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:05.665361 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:05.665289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:06.508922 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:06.508890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:06.509090 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:06.508989    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:14:06.672679 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:06.672643    2577 generic.go:356] "Generic (PLEG): container finished" podID="23e93a3f-c550-4fbb-9ef5-80fd6a249da6" containerID="33bfcafcd645e01cf128ef7f073675d41f14a657002eaf63542e3200bd77dba9" exitCode=0
Jul 07 13:14:06.673077 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:06.672737    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerDied","Data":"33bfcafcd645e01cf128ef7f073675d41f14a657002eaf63542e3200bd77dba9"}
Jul 07 13:14:07.508117 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:07.508087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:07.508217 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:07.508126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:07.508273 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:07.508218    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:14:07.508367 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:07.508343    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:14:07.676804 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:07.676652    2577 generic.go:356] "Generic (PLEG): container finished" podID="23e93a3f-c550-4fbb-9ef5-80fd6a249da6" containerID="4a67475339573027932424cd3ce839860c86ab3e6686a921939812922dfc2011" exitCode=0
Jul 07 13:14:07.676804 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:07.676729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerDied","Data":"4a67475339573027932424cd3ce839860c86ab3e6686a921939812922dfc2011"}
Jul 07 13:14:08.508682 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:08.508654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:08.508779 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:08.508759    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:14:08.681066 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:08.681004    2577 generic.go:356] "Generic (PLEG): container finished" podID="23e93a3f-c550-4fbb-9ef5-80fd6a249da6" containerID="182f3b254674df697287dffc6e17ed494afdcbdf39d8464f14891d4c7da53fd1" exitCode=0
Jul 07 13:14:08.681066 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:08.681057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerDied","Data":"182f3b254674df697287dffc6e17ed494afdcbdf39d8464f14891d4c7da53fd1"}
Jul 07 13:14:08.796294 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:08.796256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:08.869200 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:08.869173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:09.298564 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:09.298532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:09.298724 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:09.298594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:09.298724 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.298670    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 13:14:09.298798 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.298727    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret podName:040975fe-ab93-4030-9fe7-1fbf8b92a86a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:41.298712393 +0000 UTC m=+66.315868553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret") pod "global-pull-secret-syncer-889k8" (UID: "040975fe-ab93-4030-9fe7-1fbf8b92a86a") : object "kube-system"/"original-pull-secret" not registered
Jul 07 13:14:09.298798 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.298676    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:14:09.298798 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.298763    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs podName:62f5f556-db8b-4d78-a246-7e68a809d40f nodeName:}" failed. No retries permitted until 2026-07-07 13:14:41.298754878 +0000 UTC m=+66.315911039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs") pod "network-metrics-daemon-ndrqj" (UID: "62f5f556-db8b-4d78-a246-7e68a809d40f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 13:14:09.399361 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:09.399322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:09.399508 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.399488    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 13:14:09.399594 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.399516    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 13:14:09.399594 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.399529    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lf2tt for pod openshift-network-diagnostics/network-check-target-d2bxr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:14:09.399687 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.399600    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt podName:fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:41.399582173 +0000 UTC m=+66.416738350 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lf2tt" (UniqueName: "kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt") pod "network-check-target-d2bxr" (UID: "fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 13:14:09.508744 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:09.508713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:09.508890 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:09.508713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:09.508890 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.508848    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d2bxr" podUID="fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73"
Jul 07 13:14:09.508968 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:09.508905    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndrqj" podUID="62f5f556-db8b-4d78-a246-7e68a809d40f"
Jul 07 13:14:10.508932 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:10.508900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:10.509380 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:10.509030    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-889k8" podUID="040975fe-ab93-4030-9fe7-1fbf8b92a86a"
Jul 07 13:14:11.273934 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.273901    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeReady"
Jul 07 13:14:11.274103 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.274043    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 13:14:11.324937 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.324910    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vx4vz"]
Jul 07 13:14:11.328988 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.328963    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.330291 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.330265    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xgwz9"]
Jul 07 13:14:11.332258 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.332231    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 13:14:11.332258 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.332257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qwjzx\""
Jul 07 13:14:11.332498 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.332480    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 13:14:11.332917 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.332897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.335701 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.335680    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 13:14:11.335825 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.335797    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 13:14:11.335908 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.335797    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 13:14:11.335995 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.335969    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbgxn\""
Jul 07 13:14:11.339662 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.339643    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vx4vz"]
Jul 07 13:14:11.354013 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.353985    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xgwz9"]
Jul 07 13:14:11.418636 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.418607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.418746 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.418645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wsfs\" (UniqueName: \"kubernetes.io/projected/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-kube-api-access-5wsfs\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.418746 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.418719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4qcj\" (UniqueName: \"kubernetes.io/projected/2fece979-1d5a-4a80-aac4-7caf620121b0-kube-api-access-l4qcj\") pod \"ingress-canary-xgwz9\" (UID: \"2fece979-1d5a-4a80-aac4-7caf620121b0\") " pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.418870 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.418767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-tmp-dir\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.418870 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.418797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fece979-1d5a-4a80-aac4-7caf620121b0-cert\") pod \"ingress-canary-xgwz9\" (UID: \"2fece979-1d5a-4a80-aac4-7caf620121b0\") " pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.418870 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.418865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-config-volume\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.508187 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.508160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:11.508751 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.508414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:11.511479 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.511217    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 13:14:11.511479 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.511323    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 13:14:11.511479 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.511375    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjddr\""
Jul 07 13:14:11.511479 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.511385    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ztml\""
Jul 07 13:14:11.511927 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.511618    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 13:14:11.519092 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.519071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fece979-1d5a-4a80-aac4-7caf620121b0-cert\") pod \"ingress-canary-xgwz9\" (UID: \"2fece979-1d5a-4a80-aac4-7caf620121b0\") " pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.519189 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.519113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-config-volume\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.519189 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.519138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.519189 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.519153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wsfs\" (UniqueName: \"kubernetes.io/projected/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-kube-api-access-5wsfs\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.519343 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.519192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4qcj\" (UniqueName: \"kubernetes.io/projected/2fece979-1d5a-4a80-aac4-7caf620121b0-kube-api-access-l4qcj\") pod \"ingress-canary-xgwz9\" (UID: \"2fece979-1d5a-4a80-aac4-7caf620121b0\") " pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.519343 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.519234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-tmp-dir\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.519529 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.519510    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-tmp-dir\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.519877 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:11.519859    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:14:11.519975 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:11.519935    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls podName:f33e8d23-8a9a-4274-8bf6-4a44cd1399c8 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:12.019916423 +0000 UTC m=+37.037072599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls") pod "dns-default-vx4vz" (UID: "f33e8d23-8a9a-4274-8bf6-4a44cd1399c8") : secret "dns-default-metrics-tls" not found
Jul 07 13:14:11.520062 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.520040    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-config-volume\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.524630 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.524583    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fece979-1d5a-4a80-aac4-7caf620121b0-cert\") pod \"ingress-canary-xgwz9\" (UID: \"2fece979-1d5a-4a80-aac4-7caf620121b0\") " pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.531249 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.531225    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wsfs\" (UniqueName: \"kubernetes.io/projected/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-kube-api-access-5wsfs\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:11.531613 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.531592    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4qcj\" (UniqueName: \"kubernetes.io/projected/2fece979-1d5a-4a80-aac4-7caf620121b0-kube-api-access-l4qcj\") pod \"ingress-canary-xgwz9\" (UID: \"2fece979-1d5a-4a80-aac4-7caf620121b0\") " pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.646688 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.646654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xgwz9"
Jul 07 13:14:11.850694 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:11.850667    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xgwz9"]
Jul 07 13:14:11.854618 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:11.854595    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fece979_1d5a_4a80_aac4_7caf620121b0.slice/crio-97afe39eee6c9554b74ba0bdbba80dfae01ca99407f621ffedf5c87d430f4f1e WatchSource:0}: Error finding container 97afe39eee6c9554b74ba0bdbba80dfae01ca99407f621ffedf5c87d430f4f1e: Status 404 returned error can't find the container with id 97afe39eee6c9554b74ba0bdbba80dfae01ca99407f621ffedf5c87d430f4f1e
Jul 07 13:14:12.022805 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:12.022775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:12.022967 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:12.022952    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:14:12.023066 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:12.023054    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls podName:f33e8d23-8a9a-4274-8bf6-4a44cd1399c8 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:13.023033917 +0000 UTC m=+38.040190095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls") pod "dns-default-vx4vz" (UID: "f33e8d23-8a9a-4274-8bf6-4a44cd1399c8") : secret "dns-default-metrics-tls" not found
Jul 07 13:14:12.507977 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:12.507949    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:12.511469 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:12.511446    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 13:14:12.693428 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:12.693390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xgwz9" event={"ID":"2fece979-1d5a-4a80-aac4-7caf620121b0","Type":"ContainerStarted","Data":"97afe39eee6c9554b74ba0bdbba80dfae01ca99407f621ffedf5c87d430f4f1e"}
Jul 07 13:14:13.029647 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:13.029609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:13.029806 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:13.029791    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:14:13.029903 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:13.029882    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls podName:f33e8d23-8a9a-4274-8bf6-4a44cd1399c8 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:15.029862626 +0000 UTC m=+40.047018791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls") pod "dns-default-vx4vz" (UID: "f33e8d23-8a9a-4274-8bf6-4a44cd1399c8") : secret "dns-default-metrics-tls" not found
Jul 07 13:14:15.045718 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.045525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:15.046144 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:15.045693    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:14:15.046144 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:15.045870    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls podName:f33e8d23-8a9a-4274-8bf6-4a44cd1399c8 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:19.045845795 +0000 UTC m=+44.063001956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls") pod "dns-default-vx4vz" (UID: "f33e8d23-8a9a-4274-8bf6-4a44cd1399c8") : secret "dns-default-metrics-tls" not found
Jul 07 13:14:15.288737 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.288707    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"]
Jul 07 13:14:15.310707 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.310405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.312186 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.312159    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"]
Jul 07 13:14:15.317126 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.317102    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-flng5\""
Jul 07 13:14:15.317252 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.317141    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 13:14:15.317587 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.317392    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 07 13:14:15.317587 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.317418    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 13:14:15.317587 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.317423    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 07 13:14:15.392224 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.392198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"]
Jul 07 13:14:15.407207 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.407184    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"]
Jul 07 13:14:15.407358 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.407334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"
Jul 07 13:14:15.410175 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.410152    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:14:15.410175 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.410170    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 07 13:14:15.410317 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.410227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xk9r5\""
Jul 07 13:14:15.421137 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.421110    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-bn89d"]
Jul 07 13:14:15.421350 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.421331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:15.424224 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.424202    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 13:14:15.424320 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.424278    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8qctq\""
Jul 07 13:14:15.424550 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.424538    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 13:14:15.424979 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.424960    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:14:15.439335 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.439315    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"]
Jul 07 13:14:15.439441 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.439352    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-bn89d"]
Jul 07 13:14:15.439441 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.439371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"]
Jul 07 13:14:15.439511 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.439442    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.442343 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.442292    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 07 13:14:15.442343 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.442331    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 07 13:14:15.442551 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.442411    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 13:14:15.442670 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.442589    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 13:14:15.442670 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.442610    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sxvw8\""
Jul 07 13:14:15.448859 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.448639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g826p\" (UniqueName: \"kubernetes.io/projected/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-kube-api-access-g826p\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.448957 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.448857    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 07 13:14:15.449048 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.448832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.449290 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.449268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.495171 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.495138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:15.553020 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.552990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w66wn\" (UniqueName: \"kubernetes.io/projected/4a90a231-23b0-42c4-802a-12f78e2403d5-kube-api-access-w66wn\") pod \"volume-data-source-validator-5d456fdbcd-cn75k\" (UID: \"4a90a231-23b0-42c4-802a-12f78e2403d5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"
Jul 07 13:14:15.553020 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25grl\" (UniqueName: \"kubernetes.io/projected/b60fd34d-12c2-4281-8282-85e5acfbf7d2-kube-api-access-25grl\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.553200 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:15.553200 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b60fd34d-12c2-4281-8282-85e5acfbf7d2-snapshots\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.553200 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60fd34d-12c2-4281-8282-85e5acfbf7d2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.553200 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g826p\" (UniqueName: \"kubernetes.io/projected/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-kube-api-access-g826p\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.553316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60fd34d-12c2-4281-8282-85e5acfbf7d2-tmp\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.553316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x29hh\" (UniqueName: \"kubernetes.io/projected/fba320ba-2b7f-4a66-80d7-5f7508992677-kube-api-access-x29hh\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:15.553316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.553316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.553442 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b60fd34d-12c2-4281-8282-85e5acfbf7d2-serving-cert\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.553442 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60fd34d-12c2-4281-8282-85e5acfbf7d2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.553442 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:15.553390    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:15.553533 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:15.553460    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls podName:ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:16.053444557 +0000 UTC m=+41.070600719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-kx4zp" (UID: "ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63") : secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:15.553931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.553914    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.564930 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.564887    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g826p\" (UniqueName: \"kubernetes.io/projected/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-kube-api-access-g826p\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:15.593057 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.593025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log" limit=1024
Jul 07 13:14:15.654086 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:15.654167 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b60fd34d-12c2-4281-8282-85e5acfbf7d2-snapshots\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654167 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60fd34d-12c2-4281-8282-85e5acfbf7d2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654246 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:15.654193    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 13:14:15.654283 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60fd34d-12c2-4281-8282-85e5acfbf7d2-tmp\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654283 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:15.654266    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls podName:fba320ba-2b7f-4a66-80d7-5f7508992677 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:16.154252412 +0000 UTC m=+41.171408592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qhjvd" (UID: "fba320ba-2b7f-4a66-80d7-5f7508992677") : secret "samples-operator-tls" not found
Jul 07 13:14:15.654380 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x29hh\" (UniqueName: \"kubernetes.io/projected/fba320ba-2b7f-4a66-80d7-5f7508992677-kube-api-access-x29hh\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:15.654380 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b60fd34d-12c2-4281-8282-85e5acfbf7d2-serving-cert\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654478 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60fd34d-12c2-4281-8282-85e5acfbf7d2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654544 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654522    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60fd34d-12c2-4281-8282-85e5acfbf7d2-tmp\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654602 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654578    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b60fd34d-12c2-4281-8282-85e5acfbf7d2-snapshots\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654602 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654589    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60fd34d-12c2-4281-8282-85e5acfbf7d2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.654701 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w66wn\" (UniqueName: \"kubernetes.io/projected/4a90a231-23b0-42c4-802a-12f78e2403d5-kube-api-access-w66wn\") pod \"volume-data-source-validator-5d456fdbcd-cn75k\" (UID: \"4a90a231-23b0-42c4-802a-12f78e2403d5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"
Jul 07 13:14:15.654701 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.654633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25grl\" (UniqueName: \"kubernetes.io/projected/b60fd34d-12c2-4281-8282-85e5acfbf7d2-kube-api-access-25grl\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.655165 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.655149    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60fd34d-12c2-4281-8282-85e5acfbf7d2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.656610 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.656590    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b60fd34d-12c2-4281-8282-85e5acfbf7d2-serving-cert\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.663794 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.663775    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25grl\" (UniqueName: \"kubernetes.io/projected/b60fd34d-12c2-4281-8282-85e5acfbf7d2-kube-api-access-25grl\") pod \"insights-operator-5c847dcfc4-bn89d\" (UID: \"b60fd34d-12c2-4281-8282-85e5acfbf7d2\") " pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:15.664102 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.664084    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x29hh\" (UniqueName: \"kubernetes.io/projected/fba320ba-2b7f-4a66-80d7-5f7508992677-kube-api-access-x29hh\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:15.664240 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.664220    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w66wn\" (UniqueName: \"kubernetes.io/projected/4a90a231-23b0-42c4-802a-12f78e2403d5-kube-api-access-w66wn\") pod \"volume-data-source-validator-5d456fdbcd-cn75k\" (UID: \"4a90a231-23b0-42c4-802a-12f78e2403d5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"
Jul 07 13:14:15.719604 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.719582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"
Jul 07 13:14:15.750185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:15.750171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-bn89d"
Jul 07 13:14:16.058754 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.058273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:16.058754 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:16.058434    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:16.058754 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:16.058501    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls podName:ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:17.058479784 +0000 UTC m=+42.075635967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-kx4zp" (UID: "ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63") : secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:16.132336 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.132309    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k"]
Jul 07 13:14:16.136020 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:16.135993    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a90a231_23b0_42c4_802a_12f78e2403d5.slice/crio-40d27206b3925e5f39bb7d366673689d95c5cf288c1b2007de8ac8d50739335c WatchSource:0}: Error finding container 40d27206b3925e5f39bb7d366673689d95c5cf288c1b2007de8ac8d50739335c: Status 404 returned error can't find the container with id 40d27206b3925e5f39bb7d366673689d95c5cf288c1b2007de8ac8d50739335c
Jul 07 13:14:16.136678 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.136653    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-bn89d"]
Jul 07 13:14:16.139779 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:16.139756    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60fd34d_12c2_4281_8282_85e5acfbf7d2.slice/crio-fb05405c56a713d5962e3ca2c9829310ded8f1458620e261735633bb1398ec14 WatchSource:0}: Error finding container fb05405c56a713d5962e3ca2c9829310ded8f1458620e261735633bb1398ec14: Status 404 returned error can't find the container with id fb05405c56a713d5962e3ca2c9829310ded8f1458620e261735633bb1398ec14
Jul 07 13:14:16.159517 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.159495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:16.159623 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:16.159611    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 13:14:16.159668 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:16.159659    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls podName:fba320ba-2b7f-4a66-80d7-5f7508992677 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:17.159644594 +0000 UTC m=+42.176800756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qhjvd" (UID: "fba320ba-2b7f-4a66-80d7-5f7508992677") : secret "samples-operator-tls" not found
Jul 07 13:14:16.305029 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.304872    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kcxzl"]
Jul 07 13:14:16.337292 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.337262    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kcxzl"]
Jul 07 13:14:16.337415 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.337353    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.340354 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.340335    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 07 13:14:16.340473 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.340456    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:14:16.340539 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.340469    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 07 13:14:16.341127 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.341113    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mw4rn\""
Jul 07 13:14:16.341884 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.341869    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 07 13:14:16.346657 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.346640    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 07 13:14:16.461939 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.461917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7c4k\" (UniqueName: \"kubernetes.io/projected/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-kube-api-access-b7c4k\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.462033 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.461959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-serving-cert\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.462033 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.461975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-trusted-ca\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.462145 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.462122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-config\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.563294 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.563243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7c4k\" (UniqueName: \"kubernetes.io/projected/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-kube-api-access-b7c4k\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.563294 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.563283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-serving-cert\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.563425 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.563300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-trusted-ca\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.563480 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.563446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-config\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.564050 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.564018    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-config\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.564225 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.564206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-trusted-ca\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.566434 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.566411    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-serving-cert\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.572935 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.572915    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7c4k\" (UniqueName: \"kubernetes.io/projected/5f1c6542-fa3e-4921-8ce5-56a97fd9186a-kube-api-access-b7c4k\") pod \"console-operator-68b757fb67-kcxzl\" (UID: \"5f1c6542-fa3e-4921-8ce5-56a97fd9186a\") " pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.648281 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.648261    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:16.704219 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.704191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k" event={"ID":"4a90a231-23b0-42c4-802a-12f78e2403d5","Type":"ContainerStarted","Data":"40d27206b3925e5f39bb7d366673689d95c5cf288c1b2007de8ac8d50739335c"}
Jul 07 13:14:16.705982 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.705939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xgwz9" event={"ID":"2fece979-1d5a-4a80-aac4-7caf620121b0","Type":"ContainerStarted","Data":"d96d81640c3049457bb227d15bd9a2baf861bc55e8be93dcfb09a15cfc929105"}
Jul 07 13:14:16.707411 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.707368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bn89d" event={"ID":"b60fd34d-12c2-4281-8282-85e5acfbf7d2","Type":"ContainerStarted","Data":"fb05405c56a713d5962e3ca2c9829310ded8f1458620e261735633bb1398ec14"}
Jul 07 13:14:16.711227 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.711202    2577 generic.go:356] "Generic (PLEG): container finished" podID="23e93a3f-c550-4fbb-9ef5-80fd6a249da6" containerID="76896ab9c8d2ce85728a1ab671fec882d8524fcd1677df6706eff5b957cee411" exitCode=0
Jul 07 13:14:16.711340 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.711245    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerDied","Data":"76896ab9c8d2ce85728a1ab671fec882d8524fcd1677df6706eff5b957cee411"}
Jul 07 13:14:16.727172 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.727124    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xgwz9" podStartSLOduration=1.5857640750000002 podStartE2EDuration="5.727106851s" totalImagesPullingTime="4.141342776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:16.725436488 +0000 UTC m=+41.742592672" watchObservedRunningTime="2026-07-07 13:14:16.727106851 +0000 UTC m=+41.744263035"
Jul 07 13:14:16.841133 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:16.840871    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kcxzl"]
Jul 07 13:14:17.068501 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.068433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:17.069166 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:17.068557    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:17.069166 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:17.068618    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls podName:ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:19.068605113 +0000 UTC m=+44.085761274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-kx4zp" (UID: "ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63") : secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:17.169300 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.169271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:17.169468 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:17.169437    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 13:14:17.169573 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:17.169519    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls podName:fba320ba-2b7f-4a66-80d7-5f7508992677 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:19.169491812 +0000 UTC m=+44.186647973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qhjvd" (UID: "fba320ba-2b7f-4a66-80d7-5f7508992677") : secret "samples-operator-tls" not found
Jul 07 13:14:17.714557 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.714518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" event={"ID":"5f1c6542-fa3e-4921-8ce5-56a97fd9186a","Type":"ContainerStarted","Data":"e8611f625efa61ecb6e515130fe3dec9aa183261edc2a72019fabf17934b2848"}
Jul 07 13:14:17.714897 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.714875    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b758c588b-2f4xn"]
Jul 07 13:14:17.717563 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.717540    2577 generic.go:356] "Generic (PLEG): container finished" podID="23e93a3f-c550-4fbb-9ef5-80fd6a249da6" containerID="97fb19fa77bf1f421f6fec3ce81e0d9186b49d8901190fc7389302e8911609a9" exitCode=0
Jul 07 13:14:17.730986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.730805    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b758c588b-2f4xn"]
Jul 07 13:14:17.731080 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.731001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerDied","Data":"97fb19fa77bf1f421f6fec3ce81e0d9186b49d8901190fc7389302e8911609a9"}
Jul 07 13:14:17.731080 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.730930    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.733745 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.733725    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s5lng\""
Jul 07 13:14:17.733880 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.733788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 13:14:17.733880 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.733830    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 13:14:17.733985 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.733947    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 13:14:17.740191 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.740170    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 13:14:17.875379 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-image-registry-private-configuration\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.875497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-certificates\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.875497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pphkm\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-kube-api-access-pphkm\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.875497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.875607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-trusted-ca\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.875672 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-installation-pull-secrets\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.875730 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-bound-sa-token\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.875827 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.875801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-ca-trust-extracted\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976266 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-image-registry-private-configuration\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976266 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-certificates\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976427 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pphkm\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-kube-api-access-pphkm\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976427 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976427 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-trusted-ca\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976567 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-installation-pull-secrets\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976567 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:17.976435    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:14:17.976567 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:17.976453    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b758c588b-2f4xn: secret "image-registry-tls" not found
Jul 07 13:14:17.976567 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-bound-sa-token\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976567 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:17.976504    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls podName:0ef6c5b1-86ce-466a-b8b6-2862eec3e90a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:18.476487041 +0000 UTC m=+43.493643221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls") pod "image-registry-6b758c588b-2f4xn" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a") : secret "image-registry-tls" not found
Jul 07 13:14:17.976767 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-ca-trust-extracted\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.976848 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976769    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-certificates\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.977017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.976996    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-ca-trust-extracted\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.977223 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.977204    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-trusted-ca\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.978999 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.978983    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-image-registry-private-configuration\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.979043 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.979024    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-installation-pull-secrets\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.985422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.985399    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-bound-sa-token\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:17.985501 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:17.985446    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pphkm\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-kube-api-access-pphkm\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:18.481061 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:18.481025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:18.481518 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:18.481185    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:14:18.481518 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:18.481205    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b758c588b-2f4xn: secret "image-registry-tls" not found
Jul 07 13:14:18.481518 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:18.481274    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls podName:0ef6c5b1-86ce-466a-b8b6-2862eec3e90a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:19.481254495 +0000 UTC m=+44.498410676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls") pod "image-registry-6b758c588b-2f4xn" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a") : secret "image-registry-tls" not found
Jul 07 13:14:19.087257 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.087198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:19.087424 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.087368    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:19.087424 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.087386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:19.087528 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.087445    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls podName:ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:23.087423701 +0000 UTC m=+48.104579865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-kx4zp" (UID: "ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63") : secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:19.087528 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.087498    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:14:19.087626 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.087548    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls podName:f33e8d23-8a9a-4274-8bf6-4a44cd1399c8 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:27.087532331 +0000 UTC m=+52.104688513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls") pod "dns-default-vx4vz" (UID: "f33e8d23-8a9a-4274-8bf6-4a44cd1399c8") : secret "dns-default-metrics-tls" not found
Jul 07 13:14:19.188596 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.188564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:19.188729 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.188690    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 13:14:19.188790 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.188758    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls podName:fba320ba-2b7f-4a66-80d7-5f7508992677 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:23.188739327 +0000 UTC m=+48.205895488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qhjvd" (UID: "fba320ba-2b7f-4a66-80d7-5f7508992677") : secret "samples-operator-tls" not found
Jul 07 13:14:19.490627 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.490558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:19.490992 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.490666    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:14:19.490992 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.490677    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b758c588b-2f4xn: secret "image-registry-tls" not found
Jul 07 13:14:19.490992 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:19.490724    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls podName:0ef6c5b1-86ce-466a-b8b6-2862eec3e90a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:21.490712331 +0000 UTC m=+46.507868492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls") pod "image-registry-6b758c588b-2f4xn" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a") : secret "image-registry-tls" not found
Jul 07 13:14:19.726098 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.726063    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" event={"ID":"23e93a3f-c550-4fbb-9ef5-80fd6a249da6","Type":"ContainerStarted","Data":"f1f649953ad70d6874dea97388a3adc3f3e460150f79398a367cc7308a764643"}
Jul 07 13:14:19.727496 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.727467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k" event={"ID":"4a90a231-23b0-42c4-802a-12f78e2403d5","Type":"ContainerStarted","Data":"441f665de1df6a3b0a45acf5e226d1fb37398cda91eb8f9a0129d4424679d901"}
Jul 07 13:14:19.728924 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.728900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bn89d" event={"ID":"b60fd34d-12c2-4281-8282-85e5acfbf7d2","Type":"ContainerStarted","Data":"ef78302b5d7f40ffaf48e44d9a44882267b4bb766289e93d0213e9c27ab0770d"}
Jul 07 13:14:19.775885 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.775836    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jjlc2" podStartSLOduration=9.029408187 podStartE2EDuration="43.775800113s" totalImagesPullingTime="34.746391926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:19.773965064 +0000 UTC m=+44.791121247" watchObservedRunningTime="2026-07-07 13:14:19.775800113 +0000 UTC m=+44.792956297"
Jul 07 13:14:19.796963 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.796924    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-cn75k" podStartSLOduration=2.291838715 podStartE2EDuration="4.796911817s" totalImagesPullingTime="2.505073102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:19.79631415 +0000 UTC m=+44.813470333" watchObservedRunningTime="2026-07-07 13:14:19.796911817 +0000 UTC m=+44.814067997"
Jul 07 13:14:19.823683 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:19.823644    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-bn89d" podStartSLOduration=1.413607856 podStartE2EDuration="4.823630952s" totalImagesPullingTime="3.410023096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:19.822614025 +0000 UTC m=+44.839770218" watchObservedRunningTime="2026-07-07 13:14:19.823630952 +0000 UTC m=+44.840787135"
Jul 07 13:14:20.736079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:20.736048    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/0.log"
Jul 07 13:14:20.736405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:20.736088    2577 generic.go:356] "Generic (PLEG): container finished" podID="5f1c6542-fa3e-4921-8ce5-56a97fd9186a" containerID="691b9551823a5ee14bfd900c4e4e119eeea9e86465a393950ba662a3082adbda" exitCode=255
Jul 07 13:14:20.736405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:20.736158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" event={"ID":"5f1c6542-fa3e-4921-8ce5-56a97fd9186a","Type":"ContainerDied","Data":"691b9551823a5ee14bfd900c4e4e119eeea9e86465a393950ba662a3082adbda"}
Jul 07 13:14:20.736405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:20.736369    2577 scope.go:117] "RemoveContainer" containerID="691b9551823a5ee14bfd900c4e4e119eeea9e86465a393950ba662a3082adbda"
Jul 07 13:14:20.995037 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:20.994845    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2mwf7"]
Jul 07 13:14:21.012530 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.012496    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2mwf7"]
Jul 07 13:14:21.012652 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.012603    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.015698 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.015675    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 13:14:21.015794 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.015743    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 13:14:21.015899 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.015883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wkhkp\""
Jul 07 13:14:21.106217 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.106193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa650b49-e34d-4432-b6a0-e7d466eda42d-crio-socket\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.106217 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.106222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.106370 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.106246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa650b49-e34d-4432-b6a0-e7d466eda42d-data-volume\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.106370 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.106261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.106370 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.106303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766v7\" (UniqueName: \"kubernetes.io/projected/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-api-access-766v7\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.207376 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.207347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa650b49-e34d-4432-b6a0-e7d466eda42d-crio-socket\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.207376 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.207380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.207563 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.207407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa650b49-e34d-4432-b6a0-e7d466eda42d-data-volume\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.207563 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.207462    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:14:21.207563 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.207520    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:21.707506499 +0000 UTC m=+46.724662660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : configmap "kube-rbac-proxy" not found
Jul 07 13:14:21.207563 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.207515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.207563 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.207539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-766v7\" (UniqueName: \"kubernetes.io/projected/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-api-access-766v7\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.207762 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.207604    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:21.207762 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.207615    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa650b49-e34d-4432-b6a0-e7d466eda42d-crio-socket\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.207762 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.207670    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:21.707653871 +0000 UTC m=+46.724810043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:21.207880 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.207847    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa650b49-e34d-4432-b6a0-e7d466eda42d-data-volume\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.218703 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.218685    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-766v7\" (UniqueName: \"kubernetes.io/projected/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-api-access-766v7\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.509932 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.509901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:21.510121 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.510066    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:14:21.510121 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.510083    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b758c588b-2f4xn: secret "image-registry-tls" not found
Jul 07 13:14:21.510191 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.510136    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls podName:0ef6c5b1-86ce-466a-b8b6-2862eec3e90a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:25.510121706 +0000 UTC m=+50.527277872 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls") pod "image-registry-6b758c588b-2f4xn" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a") : secret "image-registry-tls" not found
Jul 07 13:14:21.712496 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.712462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.712632 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.712505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:21.712632 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.712584    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:14:21.712632 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.712600    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:21.712740 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.712651    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:22.712633897 +0000 UTC m=+47.729790058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : configmap "kube-rbac-proxy" not found
Jul 07 13:14:21.712740 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.712669    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:22.71266257 +0000 UTC m=+47.729818731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:21.740348 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.740330    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:14:21.740766 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.740752    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/0.log"
Jul 07 13:14:21.740829 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.740785    2577 generic.go:356] "Generic (PLEG): container finished" podID="5f1c6542-fa3e-4921-8ce5-56a97fd9186a" containerID="f20d406332d9861d7feced1061b1e65a608fc085e8e6d7bad5ca48b6686c78db" exitCode=255
Jul 07 13:14:21.740873 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.740841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" event={"ID":"5f1c6542-fa3e-4921-8ce5-56a97fd9186a","Type":"ContainerDied","Data":"f20d406332d9861d7feced1061b1e65a608fc085e8e6d7bad5ca48b6686c78db"}
Jul 07 13:14:21.740913 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.740891    2577 scope.go:117] "RemoveContainer" containerID="691b9551823a5ee14bfd900c4e4e119eeea9e86465a393950ba662a3082adbda"
Jul 07 13:14:21.741107 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:21.741091    2577 scope.go:117] "RemoveContainer" containerID="f20d406332d9861d7feced1061b1e65a608fc085e8e6d7bad5ca48b6686c78db"
Jul 07 13:14:21.741290 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:21.741272    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kcxzl_openshift-console-operator(5f1c6542-fa3e-4921-8ce5-56a97fd9186a)\"" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" podUID="5f1c6542-fa3e-4921-8ce5-56a97fd9186a"
Jul 07 13:14:22.373987 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:22.373964    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gg9gk_520d0a12-2834-46f1-b42a-6f4a23c9037f/dns-node-resolver/0.log"
Jul 07 13:14:22.721002 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:22.720940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:22.721002 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:22.720979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:22.721159 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:22.721050    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:14:22.721159 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:22.721095    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:22.721159 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:22.721104    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:24.721090245 +0000 UTC m=+49.738246410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : configmap "kube-rbac-proxy" not found
Jul 07 13:14:22.721159 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:22.721132    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:24.721121447 +0000 UTC m=+49.738277608 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:22.744656 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:22.744635    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:14:22.745015 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:22.744956    2577 scope.go:117] "RemoveContainer" containerID="f20d406332d9861d7feced1061b1e65a608fc085e8e6d7bad5ca48b6686c78db"
Jul 07 13:14:22.745104 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:22.745086    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kcxzl_openshift-console-operator(5f1c6542-fa3e-4921-8ce5-56a97fd9186a)\"" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" podUID="5f1c6542-fa3e-4921-8ce5-56a97fd9186a"
Jul 07 13:14:23.123982 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.123954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:23.124123 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:23.124061    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:23.124123 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:23.124110    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls podName:ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:31.124097605 +0000 UTC m=+56.141253766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-kx4zp" (UID: "ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63") : secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:23.224487 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.224460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:23.224589 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:23.224548    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 13:14:23.224633 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:23.224600    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls podName:fba320ba-2b7f-4a66-80d7-5f7508992677 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:31.224586956 +0000 UTC m=+56.241743117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qhjvd" (UID: "fba320ba-2b7f-4a66-80d7-5f7508992677") : secret "samples-operator-tls" not found
Jul 07 13:14:23.415919 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.415849    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"]
Jul 07 13:14:23.445142 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.445122    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"]
Jul 07 13:14:23.445238 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.445202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"
Jul 07 13:14:23.448251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.448230    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ql9mx\""
Jul 07 13:14:23.525721 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.525697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p9l8\" (UniqueName: \"kubernetes.io/projected/b9d3c4e1-50b9-48e4-95da-67fd8ab24556-kube-api-access-6p9l8\") pod \"network-check-source-989f8b8dc-48jb9\" (UID: \"b9d3c4e1-50b9-48e4-95da-67fd8ab24556\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"
Jul 07 13:14:23.626185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.626166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p9l8\" (UniqueName: \"kubernetes.io/projected/b9d3c4e1-50b9-48e4-95da-67fd8ab24556-kube-api-access-6p9l8\") pod \"network-check-source-989f8b8dc-48jb9\" (UID: \"b9d3c4e1-50b9-48e4-95da-67fd8ab24556\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"
Jul 07 13:14:23.635512 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.635483    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p9l8\" (UniqueName: \"kubernetes.io/projected/b9d3c4e1-50b9-48e4-95da-67fd8ab24556-kube-api-access-6p9l8\") pod \"network-check-source-989f8b8dc-48jb9\" (UID: \"b9d3c4e1-50b9-48e4-95da-67fd8ab24556\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"
Jul 07 13:14:23.753981 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.753961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"
Jul 07 13:14:23.773479 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.773456    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zm2x9_0e132ef7-c007-4cec-b2f4-83ebbc940c9e/node-ca/0.log"
Jul 07 13:14:23.903441 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:23.903307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9"]
Jul 07 13:14:23.906369 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:23.906344    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d3c4e1_50b9_48e4_95da_67fd8ab24556.slice/crio-67358461f6d3cf4ee209d4634253cb401bed49797a62ab1dc22ff38e6c08f08b WatchSource:0}: Error finding container 67358461f6d3cf4ee209d4634253cb401bed49797a62ab1dc22ff38e6c08f08b: Status 404 returned error can't find the container with id 67358461f6d3cf4ee209d4634253cb401bed49797a62ab1dc22ff38e6c08f08b
Jul 07 13:14:24.373841 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:24.373800    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgwz9_2fece979-1d5a-4a80-aac4-7caf620121b0/serve-healthcheck-canary/0.log"
Jul 07 13:14:24.736753 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:24.736648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:24.736911 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:24.736802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:24.736911 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:24.736834    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:24.736911 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:24.736910    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:28.736890106 +0000 UTC m=+53.754046267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:24.737057 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:24.736911    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:14:24.737057 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:24.736964    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:28.736948761 +0000 UTC m=+53.754104925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : configmap "kube-rbac-proxy" not found
Jul 07 13:14:24.751579 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:24.751538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9" event={"ID":"b9d3c4e1-50b9-48e4-95da-67fd8ab24556","Type":"ContainerStarted","Data":"67358461f6d3cf4ee209d4634253cb401bed49797a62ab1dc22ff38e6c08f08b"}
Jul 07 13:14:25.076743 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.076710    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"]
Jul 07 13:14:25.085912 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.085889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"
Jul 07 13:14:25.089010 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.088979    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 13:14:25.089142 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.089028    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-l8kfn\""
Jul 07 13:14:25.090419 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.090394    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 13:14:25.092554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.092531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"]
Jul 07 13:14:25.140336 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.140307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d2ch\" (UniqueName: \"kubernetes.io/projected/c09aa4b4-dce8-4b9d-8457-d012f8246c50-kube-api-access-8d2ch\") pod \"migrator-d57d58cc9-pbj4w\" (UID: \"c09aa4b4-dce8-4b9d-8457-d012f8246c50\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"
Jul 07 13:14:25.240769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.240728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d2ch\" (UniqueName: \"kubernetes.io/projected/c09aa4b4-dce8-4b9d-8457-d012f8246c50-kube-api-access-8d2ch\") pod \"migrator-d57d58cc9-pbj4w\" (UID: \"c09aa4b4-dce8-4b9d-8457-d012f8246c50\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"
Jul 07 13:14:25.256660 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.256635    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d2ch\" (UniqueName: \"kubernetes.io/projected/c09aa4b4-dce8-4b9d-8457-d012f8246c50-kube-api-access-8d2ch\") pod \"migrator-d57d58cc9-pbj4w\" (UID: \"c09aa4b4-dce8-4b9d-8457-d012f8246c50\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"
Jul 07 13:14:25.398180 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.398102    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"
Jul 07 13:14:25.543570 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.543534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:25.543792 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:25.543742    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:14:25.543792 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:25.543760    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b758c588b-2f4xn: secret "image-registry-tls" not found
Jul 07 13:14:25.543997 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:25.543833    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls podName:0ef6c5b1-86ce-466a-b8b6-2862eec3e90a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:33.543802024 +0000 UTC m=+58.560958204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls") pod "image-registry-6b758c588b-2f4xn" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a") : secret "image-registry-tls" not found
Jul 07 13:14:25.559426 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.559403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w"]
Jul 07 13:14:25.563472 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:25.563449    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09aa4b4_dce8_4b9d_8457_d012f8246c50.slice/crio-e26d5497bf6ac67869fc34c03dfa970ab654e1bfbcdc9ffbe55d430eb61938e9 WatchSource:0}: Error finding container e26d5497bf6ac67869fc34c03dfa970ab654e1bfbcdc9ffbe55d430eb61938e9: Status 404 returned error can't find the container with id e26d5497bf6ac67869fc34c03dfa970ab654e1bfbcdc9ffbe55d430eb61938e9
Jul 07 13:14:25.758998 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:25.758958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w" event={"ID":"c09aa4b4-dce8-4b9d-8457-d012f8246c50","Type":"ContainerStarted","Data":"e26d5497bf6ac67869fc34c03dfa970ab654e1bfbcdc9ffbe55d430eb61938e9"}
Jul 07 13:14:26.648869 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:26.648834    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:26.649382 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:26.648895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:26.649382 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:26.649289    2577 scope.go:117] "RemoveContainer" containerID="f20d406332d9861d7feced1061b1e65a608fc085e8e6d7bad5ca48b6686c78db"
Jul 07 13:14:26.649484 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:26.649454    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kcxzl_openshift-console-operator(5f1c6542-fa3e-4921-8ce5-56a97fd9186a)\"" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" podUID="5f1c6542-fa3e-4921-8ce5-56a97fd9186a"
Jul 07 13:14:27.158503 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:27.158466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:27.158686 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:27.158636    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 13:14:27.158754 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:27.158718    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls podName:f33e8d23-8a9a-4274-8bf6-4a44cd1399c8 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:43.158695305 +0000 UTC m=+68.175851484 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls") pod "dns-default-vx4vz" (UID: "f33e8d23-8a9a-4274-8bf6-4a44cd1399c8") : secret "dns-default-metrics-tls" not found
Jul 07 13:14:28.770041 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:28.769996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9" event={"ID":"b9d3c4e1-50b9-48e4-95da-67fd8ab24556","Type":"ContainerStarted","Data":"2dbf57167a9aecebb6e82c4323352d987a5b0b3105e29488bca143a39580d406"}
Jul 07 13:14:28.771976 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:28.771951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w" event={"ID":"c09aa4b4-dce8-4b9d-8457-d012f8246c50","Type":"ContainerStarted","Data":"d9339317397e5f9c3d46255763c8faa97b02c0cc740a30793d83fb4eb8accaa3"}
Jul 07 13:14:28.772114 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:28.771997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w" event={"ID":"c09aa4b4-dce8-4b9d-8457-d012f8246c50","Type":"ContainerStarted","Data":"121a835d24964f4b7a5a2706e3237a892ff0a6ae5491d7a47e01f7e586d4da66"}
Jul 07 13:14:28.773391 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:28.773349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:28.773504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:28.773420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:28.773504 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:28.773465    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 13:14:28.773610 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:28.773527    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:36.77351008 +0000 UTC m=+61.790666261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : configmap "kube-rbac-proxy" not found
Jul 07 13:14:28.773610 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:28.773542    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:28.773610 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:28.773599    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:36.773582835 +0000 UTC m=+61.790739016 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:28.788737 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:28.788696    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-48jb9" podStartSLOduration=1.6418869630000001 podStartE2EDuration="5.788684264s" totalImagesPullingTime="4.146797301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:28.787960219 +0000 UTC m=+53.805116438" watchObservedRunningTime="2026-07-07 13:14:28.788684264 +0000 UTC m=+53.805840446"
Jul 07 13:14:28.804148 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:28.804109    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pbj4w" podStartSLOduration=1.3126326879999999 podStartE2EDuration="3.804099299s" totalImagesPullingTime="2.491466611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:28.802891518 +0000 UTC m=+53.820047722" watchObservedRunningTime="2026-07-07 13:14:28.804099299 +0000 UTC m=+53.821255481"
Jul 07 13:14:31.192064 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:31.192027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:31.192573 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:31.192198    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:31.192573 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:31.192280    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls podName:ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:47.192258118 +0000 UTC m=+72.209414282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-kx4zp" (UID: "ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63") : secret "cluster-monitoring-operator-tls" not found
Jul 07 13:14:31.292359 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:31.292327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:31.292497 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:31.292477    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 13:14:31.292551 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:31.292541    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls podName:fba320ba-2b7f-4a66-80d7-5f7508992677 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:47.29252614 +0000 UTC m=+72.309682301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qhjvd" (UID: "fba320ba-2b7f-4a66-80d7-5f7508992677") : secret "samples-operator-tls" not found
Jul 07 13:14:33.611803 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:33.611773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:33.612313 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:33.611937    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 13:14:33.612313 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:33.611960    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b758c588b-2f4xn: secret "image-registry-tls" not found
Jul 07 13:14:33.612313 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:33.612028    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls podName:0ef6c5b1-86ce-466a-b8b6-2862eec3e90a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:49.612009148 +0000 UTC m=+74.629165311 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls") pod "image-registry-6b758c588b-2f4xn" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a") : secret "image-registry-tls" not found
Jul 07 13:14:36.689097 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:36.689066    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpsjj"
Jul 07 13:14:36.837341 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:36.837311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:36.837508 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:36.837348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:36.837508 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:36.837423    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:36.837508 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:14:36.837468    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls podName:aa650b49-e34d-4432-b6a0-e7d466eda42d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:52.837454776 +0000 UTC m=+77.854610937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2mwf7" (UID: "aa650b49-e34d-4432-b6a0-e7d466eda42d") : secret "insights-runtime-extractor-tls" not found
Jul 07 13:14:36.837912 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:36.837890    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa650b49-e34d-4432-b6a0-e7d466eda42d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:41.372310 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.372277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:41.372310 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.372315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:41.375373 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.375343    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 13:14:41.375495 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.375350    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 13:14:41.384970 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.384947    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/040975fe-ab93-4030-9fe7-1fbf8b92a86a-original-pull-secret\") pod \"global-pull-secret-syncer-889k8\" (UID: \"040975fe-ab93-4030-9fe7-1fbf8b92a86a\") " pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:41.384970 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.384965    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f5f556-db8b-4d78-a246-7e68a809d40f-metrics-certs\") pod \"network-metrics-daemon-ndrqj\" (UID: \"62f5f556-db8b-4d78-a246-7e68a809d40f\") " pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:41.472604 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.472568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:41.474971 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.474954    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf2tt\" (UniqueName: \"kubernetes.io/projected/fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73-kube-api-access-lf2tt\") pod \"network-check-target-d2bxr\" (UID: \"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73\") " pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:41.509275 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.509245    2577 scope.go:117] "RemoveContainer" containerID="f20d406332d9861d7feced1061b1e65a608fc085e8e6d7bad5ca48b6686c78db"
Jul 07 13:14:41.522532 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.522512    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjddr\""
Jul 07 13:14:41.529567 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.529551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndrqj"
Jul 07 13:14:41.534247 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.534231    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ztml\""
Jul 07 13:14:41.542443 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.542419    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:41.616300 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.616233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-889k8"
Jul 07 13:14:41.672460 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.672423    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndrqj"]
Jul 07 13:14:41.687199 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.687123    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d2bxr"]
Jul 07 13:14:41.688695 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:41.688664    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2c3bf5_c24e_49a3_8cf2_4f7c226b7c73.slice/crio-55cbcc509339e700fdfd31d11b4420e05304cf5245a7085597559f04941640f3 WatchSource:0}: Error finding container 55cbcc509339e700fdfd31d11b4420e05304cf5245a7085597559f04941640f3: Status 404 returned error can't find the container with id 55cbcc509339e700fdfd31d11b4420e05304cf5245a7085597559f04941640f3
Jul 07 13:14:41.756890 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.756866    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-889k8"]
Jul 07 13:14:41.759877 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:41.759852    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040975fe_ab93_4030_9fe7_1fbf8b92a86a.slice/crio-158c045cc7c51c9f8d9b8533a6a3d6eb4e8d147ea1336e30c75d18b6fb7a832f WatchSource:0}: Error finding container 158c045cc7c51c9f8d9b8533a6a3d6eb4e8d147ea1336e30c75d18b6fb7a832f: Status 404 returned error can't find the container with id 158c045cc7c51c9f8d9b8533a6a3d6eb4e8d147ea1336e30c75d18b6fb7a832f
Jul 07 13:14:41.814885 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.814860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndrqj" event={"ID":"62f5f556-db8b-4d78-a246-7e68a809d40f","Type":"ContainerStarted","Data":"db44cb281518a019da39089f67a2794b7b97f6686fd2822edc49e1ca2e698ccc"}
Jul 07 13:14:41.816692 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.816665    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:14:41.816767 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.816752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" event={"ID":"5f1c6542-fa3e-4921-8ce5-56a97fd9186a","Type":"ContainerStarted","Data":"06cb473f6c13b3c6250a0cc86c81e9e434be06f077365058a1192c70b552d402"}
Jul 07 13:14:41.817125 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.817078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:41.817960 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.817940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-889k8" event={"ID":"040975fe-ab93-4030-9fe7-1fbf8b92a86a","Type":"ContainerStarted","Data":"158c045cc7c51c9f8d9b8533a6a3d6eb4e8d147ea1336e30c75d18b6fb7a832f"}
Jul 07 13:14:41.819423 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.819403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d2bxr" event={"ID":"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73","Type":"ContainerStarted","Data":"49c9d60978679cf8dd2285d464e7186c8e7b8625f270c9bafe2b4e969d1c3d12"}
Jul 07 13:14:41.819521 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.819439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d2bxr" event={"ID":"fc2c3bf5-c24e-49a3-8cf2-4f7c226b7c73","Type":"ContainerStarted","Data":"55cbcc509339e700fdfd31d11b4420e05304cf5245a7085597559f04941640f3"}
Jul 07 13:14:41.819521 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.819470    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:14:41.835606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.835561    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" podStartSLOduration=22.729402196 podStartE2EDuration="25.83554847s" totalImagesPullingTime="3.106146274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:41.834388574 +0000 UTC m=+66.851544761" watchObservedRunningTime="2026-07-07 13:14:41.83554847 +0000 UTC m=+66.852704714"
Jul 07 13:14:41.851716 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:41.851673    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d2bxr" podStartSLOduration=65.851656417 podStartE2EDuration="1m5.851656417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:41.851477722 +0000 UTC m=+66.868633919" watchObservedRunningTime="2026-07-07 13:14:41.851656417 +0000 UTC m=+66.868812600"
Jul 07 13:14:42.817715 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:42.817623    2577 patch_prober.go:28] interesting pod/console-operator-68b757fb67-kcxzl container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.12:8443/readyz\": context deadline exceeded" start-of-body=
Jul 07 13:14:42.817715 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:42.817695    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl" podUID="5f1c6542-fa3e-4921-8ce5-56a97fd9186a" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.12:8443/readyz\": context deadline exceeded"
Jul 07 13:14:42.932343 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:42.932318    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-kcxzl"
Jul 07 13:14:43.186220 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.186191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:43.188448 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.188427    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f33e8d23-8a9a-4274-8bf6-4a44cd1399c8-metrics-tls\") pod \"dns-default-vx4vz\" (UID: \"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8\") " pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:43.444599 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.444527    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qwjzx\""
Jul 07 13:14:43.452325 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.452292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:43.588583 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.588556    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vx4vz"]
Jul 07 13:14:43.590946 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:43.590918    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33e8d23_8a9a_4274_8bf6_4a44cd1399c8.slice/crio-53a69a8b8e8a3d111c1d17a5e960f2866472dda3c7a8c984d322153f5d066599 WatchSource:0}: Error finding container 53a69a8b8e8a3d111c1d17a5e960f2866472dda3c7a8c984d322153f5d066599: Status 404 returned error can't find the container with id 53a69a8b8e8a3d111c1d17a5e960f2866472dda3c7a8c984d322153f5d066599
Jul 07 13:14:43.828151 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.828073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vx4vz" event={"ID":"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8","Type":"ContainerStarted","Data":"53a69a8b8e8a3d111c1d17a5e960f2866472dda3c7a8c984d322153f5d066599"}
Jul 07 13:14:43.830130 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.830092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndrqj" event={"ID":"62f5f556-db8b-4d78-a246-7e68a809d40f","Type":"ContainerStarted","Data":"38d2e416e630febbb42dcf1f91cb71a1bb562c693215449755e7a8e81d6e0bbb"}
Jul 07 13:14:43.830253 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.830158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndrqj" event={"ID":"62f5f556-db8b-4d78-a246-7e68a809d40f","Type":"ContainerStarted","Data":"ca280ebd60f4a16a6bfc2ddffe0ee9315d24e23eab4b11f7d6bb9112371cb7d5"}
Jul 07 13:14:43.851266 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:43.851229    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ndrqj" podStartSLOduration=67.544253151 podStartE2EDuration="1m8.851213633s" totalImagesPullingTime="1.306960482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:43.850838191 +0000 UTC m=+68.867994396" watchObservedRunningTime="2026-07-07 13:14:43.851213633 +0000 UTC m=+68.868369815"
Jul 07 13:14:47.224185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.224114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:47.226370 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.226348    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-kx4zp\" (UID: \"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:47.325437 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.325411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:47.327393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.327374    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fba320ba-2b7f-4a66-80d7-5f7508992677-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qhjvd\" (UID: \"fba320ba-2b7f-4a66-80d7-5f7508992677\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:47.424235 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.424211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-flng5\""
Jul 07 13:14:47.431992 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.431972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"
Jul 07 13:14:47.534649 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.534626    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8qctq\""
Jul 07 13:14:47.542279 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.542249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"
Jul 07 13:14:47.553441 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.553420    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp"]
Jul 07 13:14:47.555906 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:47.555884    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6a6cc1_2d01_4e7c_b83d_4d11f6033b63.slice/crio-a3338889fca1fa74e3ef6046903fe0f187513d7f2927fdbd9a5c810f9c9319e7 WatchSource:0}: Error finding container a3338889fca1fa74e3ef6046903fe0f187513d7f2927fdbd9a5c810f9c9319e7: Status 404 returned error can't find the container with id a3338889fca1fa74e3ef6046903fe0f187513d7f2927fdbd9a5c810f9c9319e7
Jul 07 13:14:47.668146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.668118    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd"]
Jul 07 13:14:47.844662 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.844634    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-889k8" event={"ID":"040975fe-ab93-4030-9fe7-1fbf8b92a86a","Type":"ContainerStarted","Data":"31d3e17112f1b55dabd6a2bf30206ec8009385b1859e15847ed44e6187e886b2"}
Jul 07 13:14:47.845831 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.845785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp" event={"ID":"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63","Type":"ContainerStarted","Data":"a3338889fca1fa74e3ef6046903fe0f187513d7f2927fdbd9a5c810f9c9319e7"}
Jul 07 13:14:47.847632 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.847606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vx4vz" event={"ID":"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8","Type":"ContainerStarted","Data":"0c07b7c468bda43fd13a64a5a8e26018c2ecca77549dcf402a270685020d1e14"}
Jul 07 13:14:47.847753 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.847666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vx4vz" event={"ID":"f33e8d23-8a9a-4274-8bf6-4a44cd1399c8","Type":"ContainerStarted","Data":"82257b8e10814e3453bd8d736799b2423a2fffefbb6c3ce59da29582664bcb49"}
Jul 07 13:14:47.847753 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.847701    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:14:47.848808 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.848783    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd" event={"ID":"fba320ba-2b7f-4a66-80d7-5f7508992677","Type":"ContainerStarted","Data":"b7ad81c47208e332de2e9cd6fa1ebf605a2d8c0601cde89712e3d6ac0289609d"}
Jul 07 13:14:47.862506 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.862468    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-889k8" podStartSLOduration=66.787606529 podStartE2EDuration="1m11.862453629s" totalImagesPullingTime="5.0748471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:47.861859693 +0000 UTC m=+72.879015892" watchObservedRunningTime="2026-07-07 13:14:47.862453629 +0000 UTC m=+72.879609826"
Jul 07 13:14:47.879984 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:47.879945    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vx4vz" podStartSLOduration=33.641658369 podStartE2EDuration="36.879932407s" totalImagesPullingTime="3.238274038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:47.878724943 +0000 UTC m=+72.895881150" watchObservedRunningTime="2026-07-07 13:14:47.879932407 +0000 UTC m=+72.897088589"
Jul 07 13:14:49.642664 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:49.642631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:49.645172 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:49.645148    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"image-registry-6b758c588b-2f4xn\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") " pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:49.844258 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:49.844230    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s5lng\""
Jul 07 13:14:49.852005 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:49.851979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:50.892710 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:50.892674    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b758c588b-2f4xn"]
Jul 07 13:14:50.896407 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:50.896373    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef6c5b1_86ce_466a_b8b6_2862eec3e90a.slice/crio-cdb91ede3ad2cf2d0591bc3ef94ca8bb497b79df19887877d8154125f81abc76 WatchSource:0}: Error finding container cdb91ede3ad2cf2d0591bc3ef94ca8bb497b79df19887877d8154125f81abc76: Status 404 returned error can't find the container with id cdb91ede3ad2cf2d0591bc3ef94ca8bb497b79df19887877d8154125f81abc76
Jul 07 13:14:51.867086 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.867048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd" event={"ID":"fba320ba-2b7f-4a66-80d7-5f7508992677","Type":"ContainerStarted","Data":"df95391a80600ac5121ca2dcca31beb2e3aa80acff55be7a4f7f60d20ea8dab0"}
Jul 07 13:14:51.867258 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.867115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd" event={"ID":"fba320ba-2b7f-4a66-80d7-5f7508992677","Type":"ContainerStarted","Data":"7e1149958d1038a685240e31e073379444c6efa37b74e524d575fd2c2b1da106"}
Jul 07 13:14:51.868645 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.868601    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp" event={"ID":"ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63","Type":"ContainerStarted","Data":"cb21007ec8a6c2457bd612246db423c39f0c08c3176480a0ec4b77575075b8af"}
Jul 07 13:14:51.870003 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.869982    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn" event={"ID":"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a","Type":"ContainerStarted","Data":"0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef"}
Jul 07 13:14:51.870088 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.870017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn" event={"ID":"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a","Type":"ContainerStarted","Data":"cdb91ede3ad2cf2d0591bc3ef94ca8bb497b79df19887877d8154125f81abc76"}
Jul 07 13:14:51.870088 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.870045    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:14:51.887483 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.887448    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qhjvd" podStartSLOduration=33.866683987 podStartE2EDuration="36.887435213s" totalImagesPullingTime="3.020751226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:51.887111843 +0000 UTC m=+76.904268043" watchObservedRunningTime="2026-07-07 13:14:51.887435213 +0000 UTC m=+76.904591440"
Jul 07 13:14:51.905310 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.905273    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-kx4zp" podStartSLOduration=33.733672422 podStartE2EDuration="36.905257671s" totalImagesPullingTime="3.171585249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:51.904989912 +0000 UTC m=+76.922146115" watchObservedRunningTime="2026-07-07 13:14:51.905257671 +0000 UTC m=+76.922413847"
Jul 07 13:14:51.926277 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:51.926241    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn" podStartSLOduration=34.926230176 podStartE2EDuration="34.926230176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:51.925983818 +0000 UTC m=+76.943140026" watchObservedRunningTime="2026-07-07 13:14:51.926230176 +0000 UTC m=+76.943386359"
Jul 07 13:14:52.866350 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:52.866314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:52.868571 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:52.868536    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa650b49-e34d-4432-b6a0-e7d466eda42d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2mwf7\" (UID: \"aa650b49-e34d-4432-b6a0-e7d466eda42d\") " pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:53.045312 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.045282    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xprts"]
Jul 07 13:14:53.048633 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.048614    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.052983 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.052961    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"]
Jul 07 13:14:53.054279 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.054260    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 13:14:53.055531 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.055513    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 13:14:53.055653 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.055515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-szwhl\""
Jul 07 13:14:53.055718 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.055704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.058207 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.058186    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 13:14:53.058504 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.058485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 13:14:53.058591 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.058574    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 13:14:53.059126 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.059106    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 13:14:53.059126 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.059125    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 13:14:53.059857 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.059762    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 13:14:53.060614 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.060598    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 13:14:53.064559 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.064542    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xprts"]
Jul 07 13:14:53.085641 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.085586    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"]
Jul 07 13:14:53.139509 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.139450    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wkhkp\""
Jul 07 13:14:53.141137 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.141118    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b758c588b-2f4xn"]
Jul 07 13:14:53.141955 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.141937    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2mwf7"
Jul 07 13:14:53.168693 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.168785 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mskhl\" (UniqueName: \"kubernetes.io/projected/5aaa2953-cb2b-406d-936a-8597b0992c77-kube-api-access-mskhl\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.168785 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-ca\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.168785 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a814897f-909e-4249-bf67-6ab9e4c72ece-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xprts\" (UID: \"a814897f-909e-4249-bf67-6ab9e4c72ece\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.168931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a814897f-909e-4249-bf67-6ab9e4c72ece-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xprts\" (UID: \"a814897f-909e-4249-bf67-6ab9e4c72ece\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.168931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aaa2953-cb2b-406d-936a-8597b0992c77-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.168931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-hub\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.169028 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.168947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.266322 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.266294    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65558d4569-cmj2q"]
Jul 07 13:14:53.270977 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.270941    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.271108 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mskhl\" (UniqueName: \"kubernetes.io/projected/5aaa2953-cb2b-406d-936a-8597b0992c77-kube-api-access-mskhl\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.271108 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-ca\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.271216 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a814897f-909e-4249-bf67-6ab9e4c72ece-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xprts\" (UID: \"a814897f-909e-4249-bf67-6ab9e4c72ece\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.271216 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a814897f-909e-4249-bf67-6ab9e4c72ece-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xprts\" (UID: \"a814897f-909e-4249-bf67-6ab9e4c72ece\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.271707 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aaa2953-cb2b-406d-936a-8597b0992c77-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.271795 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-hub\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.271795 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.272079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.271999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.272405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.272375    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a814897f-909e-4249-bf67-6ab9e4c72ece-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xprts\" (UID: \"a814897f-909e-4249-bf67-6ab9e4c72ece\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.272492 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.272447    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aaa2953-cb2b-406d-936a-8597b0992c77-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.274375 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.274346    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-ca\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.275108 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.274981    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-hub\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.275108 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.275089    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.275248 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.275090    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5aaa2953-cb2b-406d-936a-8597b0992c77-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.275248 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.275089    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a814897f-909e-4249-bf67-6ab9e4c72ece-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xprts\" (UID: \"a814897f-909e-4249-bf67-6ab9e4c72ece\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.287201 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.287182    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2mwf7"]
Jul 07 13:14:53.289865 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:53.289844    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa650b49_e34d_4432_b6a0_e7d466eda42d.slice/crio-444a75008efdb4a162dce33a8157618c4fd677d2188d29ad102381c39aaff357 WatchSource:0}: Error finding container 444a75008efdb4a162dce33a8157618c4fd677d2188d29ad102381c39aaff357: Status 404 returned error can't find the container with id 444a75008efdb4a162dce33a8157618c4fd677d2188d29ad102381c39aaff357
Jul 07 13:14:53.295508 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.295488    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65558d4569-cmj2q"]
Jul 07 13:14:53.297288 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.297242    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mskhl\" (UniqueName: \"kubernetes.io/projected/5aaa2953-cb2b-406d-936a-8597b0992c77-kube-api-access-mskhl\") pod \"cluster-proxy-proxy-agent-57d7656b9f-7gzdr\" (UID: \"5aaa2953-cb2b-406d-936a-8597b0992c77\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.358146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.358119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts"
Jul 07 13:14:53.373419 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0961aaf9-b89c-44dd-ad00-85dd7688e07b-installation-pull-secrets\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.373514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pd6v\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-kube-api-access-9pd6v\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.373514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0961aaf9-b89c-44dd-ad00-85dd7688e07b-trusted-ca\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.373618 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-bound-sa-token\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.373670 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0961aaf9-b89c-44dd-ad00-85dd7688e07b-image-registry-private-configuration\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.373723 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0961aaf9-b89c-44dd-ad00-85dd7688e07b-ca-trust-extracted\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.373777 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0961aaf9-b89c-44dd-ad00-85dd7688e07b-registry-certificates\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.373777 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.373748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-registry-tls\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.376323 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.376291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"
Jul 07 13:14:53.474290 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0961aaf9-b89c-44dd-ad00-85dd7688e07b-image-registry-private-configuration\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474436 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0961aaf9-b89c-44dd-ad00-85dd7688e07b-ca-trust-extracted\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474436 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0961aaf9-b89c-44dd-ad00-85dd7688e07b-registry-certificates\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474436 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-registry-tls\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474436 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0961aaf9-b89c-44dd-ad00-85dd7688e07b-installation-pull-secrets\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474436 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pd6v\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-kube-api-access-9pd6v\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474889 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0961aaf9-b89c-44dd-ad00-85dd7688e07b-trusted-ca\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474889 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-bound-sa-token\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.474889 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.474731    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0961aaf9-b89c-44dd-ad00-85dd7688e07b-ca-trust-extracted\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.475768 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.475742    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0961aaf9-b89c-44dd-ad00-85dd7688e07b-registry-certificates\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.476000 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.475977    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0961aaf9-b89c-44dd-ad00-85dd7688e07b-trusted-ca\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.478051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.478011    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-registry-tls\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.478244 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.478217    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0961aaf9-b89c-44dd-ad00-85dd7688e07b-installation-pull-secrets\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.478388 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.478371    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0961aaf9-b89c-44dd-ad00-85dd7688e07b-image-registry-private-configuration\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.485375 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.485350    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pd6v\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-kube-api-access-9pd6v\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.485529 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.485510    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0961aaf9-b89c-44dd-ad00-85dd7688e07b-bound-sa-token\") pod \"image-registry-65558d4569-cmj2q\" (UID: \"0961aaf9-b89c-44dd-ad00-85dd7688e07b\") " pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.499349 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.499330    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xprts"]
Jul 07 13:14:53.501675 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:53.501649    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda814897f_909e_4249_bf67_6ab9e4c72ece.slice/crio-d860e12a12a617f0f2fa05ecc042987a7484c69bcac0b1aa4595f6a2df1a8847 WatchSource:0}: Error finding container d860e12a12a617f0f2fa05ecc042987a7484c69bcac0b1aa4595f6a2df1a8847: Status 404 returned error can't find the container with id d860e12a12a617f0f2fa05ecc042987a7484c69bcac0b1aa4595f6a2df1a8847
Jul 07 13:14:53.525090 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.525069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr"]
Jul 07 13:14:53.527651 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:53.527615    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5aaa2953_cb2b_406d_936a_8597b0992c77.slice/crio-27a06ac8f9a2f74d5ca039784cdb98745b8add8a93ce9484499bc7e38793038e WatchSource:0}: Error finding container 27a06ac8f9a2f74d5ca039784cdb98745b8add8a93ce9484499bc7e38793038e: Status 404 returned error can't find the container with id 27a06ac8f9a2f74d5ca039784cdb98745b8add8a93ce9484499bc7e38793038e
Jul 07 13:14:53.590422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.590399    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.751223 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.751151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65558d4569-cmj2q"]
Jul 07 13:14:53.754376 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:14:53.754352    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0961aaf9_b89c_44dd_ad00_85dd7688e07b.slice/crio-7a1eeaa08527044f1d2ea2531995cc745377e6b0c343c78cdbdb1e26bf18c7ff WatchSource:0}: Error finding container 7a1eeaa08527044f1d2ea2531995cc745377e6b0c343c78cdbdb1e26bf18c7ff: Status 404 returned error can't find the container with id 7a1eeaa08527044f1d2ea2531995cc745377e6b0c343c78cdbdb1e26bf18c7ff
Jul 07 13:14:53.883768 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.883735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65558d4569-cmj2q" event={"ID":"0961aaf9-b89c-44dd-ad00-85dd7688e07b","Type":"ContainerStarted","Data":"0d76fa9d962aae2a90954b6205505c06f670b10d3c984363a482d0ba4be5ccfb"}
Jul 07 13:14:53.883925 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.883795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65558d4569-cmj2q" event={"ID":"0961aaf9-b89c-44dd-ad00-85dd7688e07b","Type":"ContainerStarted","Data":"7a1eeaa08527044f1d2ea2531995cc745377e6b0c343c78cdbdb1e26bf18c7ff"}
Jul 07 13:14:53.884693 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.884637    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:14:53.886383 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.886323    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr" event={"ID":"5aaa2953-cb2b-406d-936a-8597b0992c77","Type":"ContainerStarted","Data":"27a06ac8f9a2f74d5ca039784cdb98745b8add8a93ce9484499bc7e38793038e"}
Jul 07 13:14:53.888471 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.888279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts" event={"ID":"a814897f-909e-4249-bf67-6ab9e4c72ece","Type":"ContainerStarted","Data":"d860e12a12a617f0f2fa05ecc042987a7484c69bcac0b1aa4595f6a2df1a8847"}
Jul 07 13:14:53.890316 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.890075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mwf7" event={"ID":"aa650b49-e34d-4432-b6a0-e7d466eda42d","Type":"ContainerStarted","Data":"444a75008efdb4a162dce33a8157618c4fd677d2188d29ad102381c39aaff357"}
Jul 07 13:14:53.910156 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:53.910057    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65558d4569-cmj2q" podStartSLOduration=0.910038216 podStartE2EDuration="910.038216ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:53.908721394 +0000 UTC m=+78.925877579" watchObservedRunningTime="2026-07-07 13:14:53.910038216 +0000 UTC m=+78.927194402"
Jul 07 13:14:57.857427 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:14:57.857391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vx4vz"
Jul 07 13:15:00.918863 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.918770    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr" event={"ID":"5aaa2953-cb2b-406d-936a-8597b0992c77","Type":"ContainerStarted","Data":"6bc8a15b90f6139e2ee3700c20dfa6eb072a0b494c4341582be2ca15c895ed1f"}
Jul 07 13:15:00.918863 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.918836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr" event={"ID":"5aaa2953-cb2b-406d-936a-8597b0992c77","Type":"ContainerStarted","Data":"70c934e6cde2aa26c7940f8cf67b114b38d3f1daf069d2a69823ac5330250f8a"}
Jul 07 13:15:00.918863 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.918856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr" event={"ID":"5aaa2953-cb2b-406d-936a-8597b0992c77","Type":"ContainerStarted","Data":"3488f8b88c0ee334a688aeff71dcab20f06017b80708ad214861169f790495f3"}
Jul 07 13:15:00.920251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.920222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts" event={"ID":"a814897f-909e-4249-bf67-6ab9e4c72ece","Type":"ContainerStarted","Data":"f992b026e1138967ccfcb5b37e43cb5e506afc6b1ccd17adc9b962d5e08af796"}
Jul 07 13:15:00.922004 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.921980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mwf7" event={"ID":"aa650b49-e34d-4432-b6a0-e7d466eda42d","Type":"ContainerStarted","Data":"a19f0a583ab1d46d63cdc85664009aa7d4b983012d32250de383959eaacdd1a0"}
Jul 07 13:15:00.922089 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.922024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mwf7" event={"ID":"aa650b49-e34d-4432-b6a0-e7d466eda42d","Type":"ContainerStarted","Data":"9d30a7c4d9206613a6d3629f36f07950e9e2c5f1e8e828dac47a6ac1257220e1"}
Jul 07 13:15:00.954730 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.954686    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d7656b9f-7gzdr" podStartSLOduration=1.57117239 podStartE2EDuration="7.954672848s" totalImagesPullingTime="6.383500458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:00.953014925 +0000 UTC m=+85.970171119" watchObservedRunningTime="2026-07-07 13:15:00.954672848 +0000 UTC m=+85.971829008"
Jul 07 13:15:00.987328 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:00.987289    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xprts" podStartSLOduration=1.6193886929999999 podStartE2EDuration="7.987276048s" totalImagesPullingTime="6.367887355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:00.986718943 +0000 UTC m=+86.003875126" watchObservedRunningTime="2026-07-07 13:15:00.987276048 +0000 UTC m=+86.004432230"
Jul 07 13:15:03.938052 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:03.938016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2mwf7" event={"ID":"aa650b49-e34d-4432-b6a0-e7d466eda42d","Type":"ContainerStarted","Data":"9d44f5536c80303c008a855f287da5187137c951200a909cd7858d54b89d7851"}
Jul 07 13:15:03.962407 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:03.962330    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2mwf7" podStartSLOduration=34.238281417 podStartE2EDuration="43.962314729s" totalImagesPullingTime="9.724033312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:03.962285263 +0000 UTC m=+88.979441464" watchObservedRunningTime="2026-07-07 13:15:03.962314729 +0000 UTC m=+88.979470913"
Jul 07 13:15:10.919718 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:10.919685    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zg7jd"]
Jul 07 13:15:10.924088 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:10.924070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:10.932963 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:10.932945    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 13:15:10.937593 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:10.937573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 13:15:10.937593 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:10.937589    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h2gvw\""
Jul 07 13:15:10.937738 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:10.937590    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 13:15:10.937738 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:10.937712    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 13:15:11.011671 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-tls\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.011842 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.011842 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-sys\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.011842 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vq9n\" (UniqueName: \"kubernetes.io/projected/d22bfda0-4e00-4dac-8da9-731834d7f5d4-kube-api-access-7vq9n\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.011842 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-root\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.012079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-textfile\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.012079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-wtmp\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.012079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.011998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.012079 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.012027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22bfda0-4e00-4dac-8da9-731834d7f5d4-metrics-client-ca\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113360 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-root\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113360 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-textfile\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113570 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-wtmp\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113570 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113570 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113458    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-root\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113570 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22bfda0-4e00-4dac-8da9-731834d7f5d4-metrics-client-ca\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-tls\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-sys\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113529    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-wtmp\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.113769 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:15:11.113680    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 13:15:11.113769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vq9n\" (UniqueName: \"kubernetes.io/projected/d22bfda0-4e00-4dac-8da9-731834d7f5d4-kube-api-access-7vq9n\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.114081 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:15:11.113775    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-tls podName:d22bfda0-4e00-4dac-8da9-731834d7f5d4 nodeName:}" failed. No retries permitted until 2026-07-07 13:15:11.613747981 +0000 UTC m=+96.630904150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-tls") pod "node-exporter-zg7jd" (UID: "d22bfda0-4e00-4dac-8da9-731834d7f5d4") : secret "node-exporter-tls" not found
Jul 07 13:15:11.114081 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d22bfda0-4e00-4dac-8da9-731834d7f5d4-sys\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.114081 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.113868    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-textfile\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.114081 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.114006    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-accelerators-collector-config\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.114081 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.114049    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d22bfda0-4e00-4dac-8da9-731834d7f5d4-metrics-client-ca\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.115986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.115965    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.124286 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.124264    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vq9n\" (UniqueName: \"kubernetes.io/projected/d22bfda0-4e00-4dac-8da9-731834d7f5d4-kube-api-access-7vq9n\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.618017 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.617982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-tls\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.620228 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.620206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d22bfda0-4e00-4dac-8da9-731834d7f5d4-node-exporter-tls\") pod \"node-exporter-zg7jd\" (UID: \"d22bfda0-4e00-4dac-8da9-731834d7f5d4\") " pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.832502 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.832473    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zg7jd"
Jul 07 13:15:11.845164 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:15:11.845141    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22bfda0_4e00_4dac_8da9_731834d7f5d4.slice/crio-649437ddf419391c2c9a47e6d87cedf477e2c42728190b523ad9ed15f9e33f64 WatchSource:0}: Error finding container 649437ddf419391c2c9a47e6d87cedf477e2c42728190b523ad9ed15f9e33f64: Status 404 returned error can't find the container with id 649437ddf419391c2c9a47e6d87cedf477e2c42728190b523ad9ed15f9e33f64
Jul 07 13:15:11.968060 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:11.967981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zg7jd" event={"ID":"d22bfda0-4e00-4dac-8da9-731834d7f5d4","Type":"ContainerStarted","Data":"649437ddf419391c2c9a47e6d87cedf477e2c42728190b523ad9ed15f9e33f64"}
Jul 07 13:15:12.826390 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:12.826361    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d2bxr"
Jul 07 13:15:13.895147 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:13.895119    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:15:13.978108 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:13.978075    2577 generic.go:356] "Generic (PLEG): container finished" podID="d22bfda0-4e00-4dac-8da9-731834d7f5d4" containerID="3d944a44c9470572ba8b266e46eebdeba6d9fe35e192f1e51a854c4909053cb9" exitCode=0
Jul 07 13:15:13.978268 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:13.978126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zg7jd" event={"ID":"d22bfda0-4e00-4dac-8da9-731834d7f5d4","Type":"ContainerDied","Data":"3d944a44c9470572ba8b266e46eebdeba6d9fe35e192f1e51a854c4909053cb9"}
Jul 07 13:15:14.983492 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:14.983461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zg7jd" event={"ID":"d22bfda0-4e00-4dac-8da9-731834d7f5d4","Type":"ContainerStarted","Data":"30d31a6d2f5b438531dc6755a6b8c444f605e484b1b99013370ff6d626185799"}
Jul 07 13:15:14.983859 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:14.983508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zg7jd" event={"ID":"d22bfda0-4e00-4dac-8da9-731834d7f5d4","Type":"ContainerStarted","Data":"69736a9dfcd8a093b65a6f08fd3c897cb15d491cb0c108cfc8b511a09007faff"}
Jul 07 13:15:15.903173 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:15.903139    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65558d4569-cmj2q"
Jul 07 13:15:15.928804 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:15.928750    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zg7jd" podStartSLOduration=4.53498333 podStartE2EDuration="5.928734656s" totalImagesPullingTime="1.331920523s" totalInitContainerRuntime="61.830803ms" isStatefulPod=true podCreationTimestamp="2026-07-07 13:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:15.025053983 +0000 UTC m=+100.042210166" watchObservedRunningTime="2026-07-07 13:15:15.928734656 +0000 UTC m=+100.945890886"
Jul 07 13:15:18.922476 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:18.922437    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn" podUID="0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" containerName="registry" containerID="cri-o://0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef" gracePeriod=30
Jul 07 13:15:19.167037 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.167013    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:15:19.275003 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.274973    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-ca-trust-extracted\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275150 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275018    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-installation-pull-secrets\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275150 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275043    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275150 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-trusted-ca\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275150 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-certificates\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275150 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275124    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-bound-sa-token\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275150 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275149    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-image-registry-private-configuration\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275445 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275172    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pphkm\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-kube-api-access-pphkm\") pod \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\" (UID: \"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a\") "
Jul 07 13:15:19.275574 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275544    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:15:19.275743 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.275558    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:15:19.277543 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.277514    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:15:19.277674 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.277648    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-kube-api-access-pphkm" (OuterVolumeSpecName: "kube-api-access-pphkm") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "kube-api-access-pphkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:15:19.277793 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.277768    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:15:19.277901 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.277882    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:15:19.277965 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.277944    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:15:19.284213 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.284189    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" (UID: "0ef6c5b1-86ce-466a-b8b6-2862eec3e90a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:15:19.375725 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375700    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-installation-pull-secrets\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:19.375856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375729    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-tls\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:19.375856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375742    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-trusted-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:19.375856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375750    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-registry-certificates\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:19.375856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375758    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-bound-sa-token\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:19.375856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375768    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-image-registry-private-configuration\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:19.375856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375777    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pphkm\" (UniqueName: \"kubernetes.io/projected/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-kube-api-access-pphkm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:19.375856 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:19.375786    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a-ca-trust-extracted\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:15:20.002838 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.002778    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" containerID="0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef" exitCode=0
Jul 07 13:15:20.003270 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.002873    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn"
Jul 07 13:15:20.003270 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.002870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn" event={"ID":"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a","Type":"ContainerDied","Data":"0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef"}
Jul 07 13:15:20.003270 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.003024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b758c588b-2f4xn" event={"ID":"0ef6c5b1-86ce-466a-b8b6-2862eec3e90a","Type":"ContainerDied","Data":"cdb91ede3ad2cf2d0591bc3ef94ca8bb497b79df19887877d8154125f81abc76"}
Jul 07 13:15:20.003270 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.003058    2577 scope.go:117] "RemoveContainer" containerID="0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef"
Jul 07 13:15:20.010947 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.010931    2577 scope.go:117] "RemoveContainer" containerID="0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef"
Jul 07 13:15:20.011202 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:15:20.011184    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef\": container with ID starting with 0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef not found: ID does not exist" containerID="0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef"
Jul 07 13:15:20.011270 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.011208    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef"} err="failed to get container status \"0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef\": rpc error: code = NotFound desc = could not find container \"0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef\": container with ID starting with 0024e544cf38775158c3eff1410de7bf1ac7a9b57f94c83b7e181c3cef8b5bef not found: ID does not exist"
Jul 07 13:15:20.028012 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.027990    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b758c588b-2f4xn"]
Jul 07 13:15:20.035634 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:20.035617    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b758c588b-2f4xn"]
Jul 07 13:15:21.513133 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:21.513096    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" path="/var/lib/kubelet/pods/0ef6c5b1-86ce-466a-b8b6-2862eec3e90a/volumes"
Jul 07 13:15:36.060571 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:36.060536    2577 generic.go:356] "Generic (PLEG): container finished" podID="b60fd34d-12c2-4281-8282-85e5acfbf7d2" containerID="ef78302b5d7f40ffaf48e44d9a44882267b4bb766289e93d0213e9c27ab0770d" exitCode=0
Jul 07 13:15:36.060979 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:36.060582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bn89d" event={"ID":"b60fd34d-12c2-4281-8282-85e5acfbf7d2","Type":"ContainerDied","Data":"ef78302b5d7f40ffaf48e44d9a44882267b4bb766289e93d0213e9c27ab0770d"}
Jul 07 13:15:36.060979 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:36.060890    2577 scope.go:117] "RemoveContainer" containerID="ef78302b5d7f40ffaf48e44d9a44882267b4bb766289e93d0213e9c27ab0770d"
Jul 07 13:15:36.833340 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:36.833302    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgwz9_2fece979-1d5a-4a80-aac4-7caf620121b0/serve-healthcheck-canary/0.log"
Jul 07 13:15:37.066642 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:15:37.066603    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-bn89d" event={"ID":"b60fd34d-12c2-4281-8282-85e5acfbf7d2","Type":"ContainerStarted","Data":"b29b5ca37acdb637d48815be21d7e17922f263df3dee4a430e24b009119aa20f"}
Jul 07 13:18:35.398005 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:18:35.397976    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:18:35.404490 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:18:35.404468    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:18:35.407634 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:18:35.407615    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 13:19:05.257645 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.257616    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"]
Jul 07 13:19:05.259910 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.257881    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" containerName="registry"
Jul 07 13:19:05.259910 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.257893    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" containerName="registry"
Jul 07 13:19:05.259910 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.257951    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ef6c5b1-86ce-466a-b8b6-2862eec3e90a" containerName="registry"
Jul 07 13:19:05.260747 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.260733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.265382 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.265359    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 13:19:05.265510 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.265428    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ssqv6\""
Jul 07 13:19:05.265510 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.265441    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 13:19:05.265510 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.265428    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 13:19:05.265904 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.265889    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 13:19:05.287647 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.287627    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"]
Jul 07 13:19:05.405110 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.405082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a75f8992-baaf-4577-a530-73dec6dec72d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.405225 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.405119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a75f8992-baaf-4577-a530-73dec6dec72d-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.405225 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.405140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a75f8992-baaf-4577-a530-73dec6dec72d-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.405295 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.405223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a75f8992-baaf-4577-a530-73dec6dec72d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.405295 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.405248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29s6q\" (UniqueName: \"kubernetes.io/projected/a75f8992-baaf-4577-a530-73dec6dec72d-kube-api-access-29s6q\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.506427 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.506399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a75f8992-baaf-4577-a530-73dec6dec72d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.506554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.506431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29s6q\" (UniqueName: \"kubernetes.io/projected/a75f8992-baaf-4577-a530-73dec6dec72d-kube-api-access-29s6q\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.506554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.506456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a75f8992-baaf-4577-a530-73dec6dec72d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.506554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.506487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a75f8992-baaf-4577-a530-73dec6dec72d-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.506554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.506516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a75f8992-baaf-4577-a530-73dec6dec72d-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.506931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.506908    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a75f8992-baaf-4577-a530-73dec6dec72d-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.508758 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.508690    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a75f8992-baaf-4577-a530-73dec6dec72d-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.508946 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.508928    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a75f8992-baaf-4577-a530-73dec6dec72d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.509020 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.508985    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a75f8992-baaf-4577-a530-73dec6dec72d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.518308 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.518287    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29s6q\" (UniqueName: \"kubernetes.io/projected/a75f8992-baaf-4577-a530-73dec6dec72d-kube-api-access-29s6q\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-hrkm9\" (UID: \"a75f8992-baaf-4577-a530-73dec6dec72d\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.570415 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.570391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:05.704109 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.704036    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"]
Jul 07 13:19:05.706388 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:19:05.706359    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75f8992_baaf_4577_a530_73dec6dec72d.slice/crio-0042d77855e1390ba15d5fa19ff39ee6f8eee7141470281fec0b4289ede4cac6 WatchSource:0}: Error finding container 0042d77855e1390ba15d5fa19ff39ee6f8eee7141470281fec0b4289ede4cac6: Status 404 returned error can't find the container with id 0042d77855e1390ba15d5fa19ff39ee6f8eee7141470281fec0b4289ede4cac6
Jul 07 13:19:05.708186 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.708168    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:19:05.793936 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:05.793877    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9" event={"ID":"a75f8992-baaf-4577-a530-73dec6dec72d","Type":"ContainerStarted","Data":"0042d77855e1390ba15d5fa19ff39ee6f8eee7141470281fec0b4289ede4cac6"}
Jul 07 13:19:09.814057 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:09.814020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9" event={"ID":"a75f8992-baaf-4577-a530-73dec6dec72d","Type":"ContainerStarted","Data":"40b9570506fb0907ea93b6827061afb0177833893f2872d106d07acd3f8ded9f"}
Jul 07 13:19:10.819863 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:10.819805    2577 generic.go:356] "Generic (PLEG): container finished" podID="a75f8992-baaf-4577-a530-73dec6dec72d" containerID="40b9570506fb0907ea93b6827061afb0177833893f2872d106d07acd3f8ded9f" exitCode=0
Jul 07 13:19:10.820305 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:10.819869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9" event={"ID":"a75f8992-baaf-4577-a530-73dec6dec72d","Type":"ContainerDied","Data":"40b9570506fb0907ea93b6827061afb0177833893f2872d106d07acd3f8ded9f"}
Jul 07 13:19:11.824965 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:11.824933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9" event={"ID":"a75f8992-baaf-4577-a530-73dec6dec72d","Type":"ContainerStarted","Data":"4f988ff51751df904bb42cf4836ea002274ed094a7b5c1b7ce356b603505531a"}
Jul 07 13:19:11.825364 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:11.825072    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:12.329224 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:12.329164    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9" podStartSLOduration=2.795492517 podStartE2EDuration="7.329149357s" totalImagesPullingTime="3.445997785s" totalInitContainerRuntime="1.087659055s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:12.31889746 +0000 UTC m=+337.336053827" watchObservedRunningTime="2026-07-07 13:19:12.329149357 +0000 UTC m=+337.346305539"
Jul 07 13:19:22.831370 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:22.831340    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-hrkm9"
Jul 07 13:19:25.245768 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.245734    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"]
Jul 07 13:19:25.259210 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.259188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.267866 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.267844    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 13:19:25.267970 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.267873    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 13:19:25.269785 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.269763    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 13:19:25.269924 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.269768    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 13:19:25.269924 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.269769    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vr5k8\""
Jul 07 13:19:25.273688 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.273662    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"]
Jul 07 13:19:25.351209 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.351169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511ffd2b-2823-40a2-93f0-7b7670c7e011-tls-certs\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.351373 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.351223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fwpj\" (UniqueName: \"kubernetes.io/projected/511ffd2b-2823-40a2-93f0-7b7670c7e011-kube-api-access-6fwpj\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.351373 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.351314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/511ffd2b-2823-40a2-93f0-7b7670c7e011-tmp\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.417306 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.417272    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nsp4s"]
Jul 07 13:19:25.420543 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.420524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:25.423474 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.423457    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 13:19:25.423625 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.423609    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tk9pk\""
Jul 07 13:19:25.439712 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.439687    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nsp4s"]
Jul 07 13:19:25.452388 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.452363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fwpj\" (UniqueName: \"kubernetes.io/projected/511ffd2b-2823-40a2-93f0-7b7670c7e011-kube-api-access-6fwpj\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.452516 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.452402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-cert\") pod \"odh-model-controller-7998c67dc4-nsp4s\" (UID: \"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10\") " pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:25.452575 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.452526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/511ffd2b-2823-40a2-93f0-7b7670c7e011-tmp\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.452637 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.452570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsnqx\" (UniqueName: \"kubernetes.io/projected/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-kube-api-access-zsnqx\") pod \"odh-model-controller-7998c67dc4-nsp4s\" (UID: \"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10\") " pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:25.452637 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.452610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511ffd2b-2823-40a2-93f0-7b7670c7e011-tls-certs\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.454849 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.454803    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/511ffd2b-2823-40a2-93f0-7b7670c7e011-tmp\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.455104 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.455082    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511ffd2b-2823-40a2-93f0-7b7670c7e011-tls-certs\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.465707 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.465678    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fwpj\" (UniqueName: \"kubernetes.io/projected/511ffd2b-2823-40a2-93f0-7b7670c7e011-kube-api-access-6fwpj\") pod \"kube-auth-proxy-5747f4757f-4xbgr\" (UID: \"511ffd2b-2823-40a2-93f0-7b7670c7e011\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.553464 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.553431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsnqx\" (UniqueName: \"kubernetes.io/projected/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-kube-api-access-zsnqx\") pod \"odh-model-controller-7998c67dc4-nsp4s\" (UID: \"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10\") " pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:25.553653 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.553489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-cert\") pod \"odh-model-controller-7998c67dc4-nsp4s\" (UID: \"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10\") " pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:25.553653 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:19:25.553578    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 13:19:25.553653 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:19:25.553639    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-cert podName:310e156e-0ad3-429b-a2a1-cfdaa6bd7b10 nodeName:}" failed. No retries permitted until 2026-07-07 13:19:26.053620745 +0000 UTC m=+351.070776907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-cert") pod "odh-model-controller-7998c67dc4-nsp4s" (UID: "310e156e-0ad3-429b-a2a1-cfdaa6bd7b10") : secret "odh-model-controller-webhook-cert" not found
Jul 07 13:19:25.567533 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.567501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"
Jul 07 13:19:25.574702 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.574681    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsnqx\" (UniqueName: \"kubernetes.io/projected/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-kube-api-access-zsnqx\") pod \"odh-model-controller-7998c67dc4-nsp4s\" (UID: \"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10\") " pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:25.698976 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.698952    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr"]
Jul 07 13:19:25.700406 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:19:25.700380    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511ffd2b_2823_40a2_93f0_7b7670c7e011.slice/crio-186da3b871ca74d8e108d42a1efcae1c28e9f2b9705aec8dc1f819512e7cfbca WatchSource:0}: Error finding container 186da3b871ca74d8e108d42a1efcae1c28e9f2b9705aec8dc1f819512e7cfbca: Status 404 returned error can't find the container with id 186da3b871ca74d8e108d42a1efcae1c28e9f2b9705aec8dc1f819512e7cfbca
Jul 07 13:19:25.886610 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:25.886541    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr" event={"ID":"511ffd2b-2823-40a2-93f0-7b7670c7e011","Type":"ContainerStarted","Data":"186da3b871ca74d8e108d42a1efcae1c28e9f2b9705aec8dc1f819512e7cfbca"}
Jul 07 13:19:26.057806 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.057776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-cert\") pod \"odh-model-controller-7998c67dc4-nsp4s\" (UID: \"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10\") " pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:26.060091 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.060073    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/310e156e-0ad3-429b-a2a1-cfdaa6bd7b10-cert\") pod \"odh-model-controller-7998c67dc4-nsp4s\" (UID: \"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10\") " pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:26.329769 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.329743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:26.480739 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.480705    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nsp4s"]
Jul 07 13:19:26.493534 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:19:26.493487    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310e156e_0ad3_429b_a2a1_cfdaa6bd7b10.slice/crio-1feb5b6e597f3ce651fd8f819bc1041c5f73ce1edb2eab8b60c6140bd358cacc WatchSource:0}: Error finding container 1feb5b6e597f3ce651fd8f819bc1041c5f73ce1edb2eab8b60c6140bd358cacc: Status 404 returned error can't find the container with id 1feb5b6e597f3ce651fd8f819bc1041c5f73ce1edb2eab8b60c6140bd358cacc
Jul 07 13:19:26.648929 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.648843    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gnj5b"]
Jul 07 13:19:26.653656 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.653619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:26.658043 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.658007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 13:19:26.661370 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.661341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z78s8\" (UniqueName: \"kubernetes.io/projected/e13cbdde-db69-441b-beac-6d79b8a6edbe-kube-api-access-z78s8\") pod \"model-serving-api-54d89bd5fc-gnj5b\" (UID: \"e13cbdde-db69-441b-beac-6d79b8a6edbe\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:26.661475 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.661425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e13cbdde-db69-441b-beac-6d79b8a6edbe-tls-certs\") pod \"model-serving-api-54d89bd5fc-gnj5b\" (UID: \"e13cbdde-db69-441b-beac-6d79b8a6edbe\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:26.679378 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.679349    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gnj5b"]
Jul 07 13:19:26.762388 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.762352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e13cbdde-db69-441b-beac-6d79b8a6edbe-tls-certs\") pod \"model-serving-api-54d89bd5fc-gnj5b\" (UID: \"e13cbdde-db69-441b-beac-6d79b8a6edbe\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:26.762544 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.762428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z78s8\" (UniqueName: \"kubernetes.io/projected/e13cbdde-db69-441b-beac-6d79b8a6edbe-kube-api-access-z78s8\") pod \"model-serving-api-54d89bd5fc-gnj5b\" (UID: \"e13cbdde-db69-441b-beac-6d79b8a6edbe\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:26.765259 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.765233    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e13cbdde-db69-441b-beac-6d79b8a6edbe-tls-certs\") pod \"model-serving-api-54d89bd5fc-gnj5b\" (UID: \"e13cbdde-db69-441b-beac-6d79b8a6edbe\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:26.791834 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.791786    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z78s8\" (UniqueName: \"kubernetes.io/projected/e13cbdde-db69-441b-beac-6d79b8a6edbe-kube-api-access-z78s8\") pod \"model-serving-api-54d89bd5fc-gnj5b\" (UID: \"e13cbdde-db69-441b-beac-6d79b8a6edbe\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:26.891740 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.891705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s" event={"ID":"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10","Type":"ContainerStarted","Data":"1feb5b6e597f3ce651fd8f819bc1041c5f73ce1edb2eab8b60c6140bd358cacc"}
Jul 07 13:19:26.966375 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:26.966262    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:27.100901 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:27.100868    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gnj5b"]
Jul 07 13:19:27.102869 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:19:27.102841    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13cbdde_db69_441b_beac_6d79b8a6edbe.slice/crio-3d38c8b359802bb532082187744f5054ac84f70c681dd7c5adc40f75c13450e6 WatchSource:0}: Error finding container 3d38c8b359802bb532082187744f5054ac84f70c681dd7c5adc40f75c13450e6: Status 404 returned error can't find the container with id 3d38c8b359802bb532082187744f5054ac84f70c681dd7c5adc40f75c13450e6
Jul 07 13:19:27.899489 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:27.899447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b" event={"ID":"e13cbdde-db69-441b-beac-6d79b8a6edbe","Type":"ContainerStarted","Data":"3d38c8b359802bb532082187744f5054ac84f70c681dd7c5adc40f75c13450e6"}
Jul 07 13:19:30.913603 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:30.913514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s" event={"ID":"310e156e-0ad3-429b-a2a1-cfdaa6bd7b10","Type":"ContainerStarted","Data":"95c879cab0ccd9afa5e6122e03176148e0f703a075449dc08422a442009c2be4"}
Jul 07 13:19:30.914060 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:30.913606    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:30.914994 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:30.914963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b" event={"ID":"e13cbdde-db69-441b-beac-6d79b8a6edbe","Type":"ContainerStarted","Data":"c7949f665755ba80a809fb05945b46954609e09280587d9b112e6cf0ec3a27c7"}
Jul 07 13:19:30.915117 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:30.915020    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:30.916263 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:30.916240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr" event={"ID":"511ffd2b-2823-40a2-93f0-7b7670c7e011","Type":"ContainerStarted","Data":"d35abce3ccb8bd5daff025037b80b2d8d35085bb812fbe3d1e32540b296381be"}
Jul 07 13:19:30.932773 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:30.932735    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s" podStartSLOduration=2.322190399 podStartE2EDuration="5.932722966s" totalImagesPullingTime="3.610532567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:30.931776927 +0000 UTC m=+355.948933111" watchObservedRunningTime="2026-07-07 13:19:30.932722966 +0000 UTC m=+355.949879148"
Jul 07 13:19:30.951487 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:30.951446    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5747f4757f-4xbgr" podStartSLOduration=1.547348661 podStartE2EDuration="5.951433951s" totalImagesPullingTime="4.40408529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:30.94936151 +0000 UTC m=+355.966517692" watchObservedRunningTime="2026-07-07 13:19:30.951433951 +0000 UTC m=+355.968590134"
Jul 07 13:19:33.645171 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.645121    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b" podStartSLOduration=4.173247899 podStartE2EDuration="7.645104115s" totalImagesPullingTime="3.471856216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:30.966332093 +0000 UTC m=+355.983488278" watchObservedRunningTime="2026-07-07 13:19:33.645104115 +0000 UTC m=+358.662260298"
Jul 07 13:19:33.645908 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.645802    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"]
Jul 07 13:19:33.649210 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.649188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.656013 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.655991    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 13:19:33.657257 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.657234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ps5lk\""
Jul 07 13:19:33.657414 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.657393    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 13:19:33.657510 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.657450    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 13:19:33.657510 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.657362    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 13:19:33.657610 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.657351    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 13:19:33.681590 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.681565    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"]
Jul 07 13:19:33.722414 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.722385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqlnv\" (UniqueName: \"kubernetes.io/projected/1c30aac7-1055-498c-8e21-93e2a4352334-kube-api-access-fqlnv\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.722541 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.722420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c30aac7-1055-498c-8e21-93e2a4352334-metrics-cert\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.722541 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.722445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c30aac7-1055-498c-8e21-93e2a4352334-cert\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.722541 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.722464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1c30aac7-1055-498c-8e21-93e2a4352334-manager-config\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.823277 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.823248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqlnv\" (UniqueName: \"kubernetes.io/projected/1c30aac7-1055-498c-8e21-93e2a4352334-kube-api-access-fqlnv\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.823393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.823281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c30aac7-1055-498c-8e21-93e2a4352334-metrics-cert\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.823393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.823306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c30aac7-1055-498c-8e21-93e2a4352334-cert\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.823393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.823330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1c30aac7-1055-498c-8e21-93e2a4352334-manager-config\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.824064 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.824036    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1c30aac7-1055-498c-8e21-93e2a4352334-manager-config\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.825680 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.825658    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c30aac7-1055-498c-8e21-93e2a4352334-cert\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.825767 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.825726    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c30aac7-1055-498c-8e21-93e2a4352334-metrics-cert\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.848586 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.848561    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqlnv\" (UniqueName: \"kubernetes.io/projected/1c30aac7-1055-498c-8e21-93e2a4352334-kube-api-access-fqlnv\") pod \"lws-controller-manager-5b44df4f55-xkbtg\" (UID: \"1c30aac7-1055-498c-8e21-93e2a4352334\") " pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:33.959062 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:33.958995    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:34.116203 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:34.116180    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"]
Jul 07 13:19:34.118539 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:19:34.118510    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c30aac7_1055_498c_8e21_93e2a4352334.slice/crio-ee4c33582ae8ab409d85dbc79b40e8a19284b3ed3577fa187259710496978bcc WatchSource:0}: Error finding container ee4c33582ae8ab409d85dbc79b40e8a19284b3ed3577fa187259710496978bcc: Status 404 returned error can't find the container with id ee4c33582ae8ab409d85dbc79b40e8a19284b3ed3577fa187259710496978bcc
Jul 07 13:19:34.936376 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:34.936271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg" event={"ID":"1c30aac7-1055-498c-8e21-93e2a4352334","Type":"ContainerStarted","Data":"ee4c33582ae8ab409d85dbc79b40e8a19284b3ed3577fa187259710496978bcc"}
Jul 07 13:19:36.946471 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:36.946385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg" event={"ID":"1c30aac7-1055-498c-8e21-93e2a4352334","Type":"ContainerStarted","Data":"2056b8bdbd44b41bd1f1d3e09c4e30843bcb44d01044be15e2380bfaec783015"}
Jul 07 13:19:36.946846 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:36.946514    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:19:36.971206 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:36.971159    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg" podStartSLOduration=1.567926851 podStartE2EDuration="3.971145842s" totalImagesPullingTime="2.403218991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:36.969600003 +0000 UTC m=+361.986756189" watchObservedRunningTime="2026-07-07 13:19:36.971145842 +0000 UTC m=+361.988302043"
Jul 07 13:19:41.922708 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:41.922679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nsp4s"
Jul 07 13:19:41.924643 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:41.924619    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gnj5b"
Jul 07 13:19:47.953743 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:19:47.953718    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b44df4f55-xkbtg"
Jul 07 13:22:20.784998 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.784964    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:22:20.788056 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.788039    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:20.791033 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.791012    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 13:22:20.791155 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.791018    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 13:22:20.791155 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.791018    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7jgsl\""
Jul 07 13:22:20.792672 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.792656    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 13:22:20.798391 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.798373    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:22:20.827911 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.827887    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:22:20.943116 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.943088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3f1f12a-5579-4eb6-be7c-7ee8140abf96-config-file\") pod \"limitador-limitador-54d6d988d5-vq6t9\" (UID: \"f3f1f12a-5579-4eb6-be7c-7ee8140abf96\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:20.943233 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:20.943137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjjk8\" (UniqueName: \"kubernetes.io/projected/f3f1f12a-5579-4eb6-be7c-7ee8140abf96-kube-api-access-fjjk8\") pod \"limitador-limitador-54d6d988d5-vq6t9\" (UID: \"f3f1f12a-5579-4eb6-be7c-7ee8140abf96\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:21.043767 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.043710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3f1f12a-5579-4eb6-be7c-7ee8140abf96-config-file\") pod \"limitador-limitador-54d6d988d5-vq6t9\" (UID: \"f3f1f12a-5579-4eb6-be7c-7ee8140abf96\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:21.043767 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.043751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjjk8\" (UniqueName: \"kubernetes.io/projected/f3f1f12a-5579-4eb6-be7c-7ee8140abf96-kube-api-access-fjjk8\") pod \"limitador-limitador-54d6d988d5-vq6t9\" (UID: \"f3f1f12a-5579-4eb6-be7c-7ee8140abf96\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:21.044398 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.044378    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3f1f12a-5579-4eb6-be7c-7ee8140abf96-config-file\") pod \"limitador-limitador-54d6d988d5-vq6t9\" (UID: \"f3f1f12a-5579-4eb6-be7c-7ee8140abf96\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:21.068413 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.068388    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjjk8\" (UniqueName: \"kubernetes.io/projected/f3f1f12a-5579-4eb6-be7c-7ee8140abf96-kube-api-access-fjjk8\") pod \"limitador-limitador-54d6d988d5-vq6t9\" (UID: \"f3f1f12a-5579-4eb6-be7c-7ee8140abf96\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:21.098256 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.098236    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:21.225922 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.225898    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:22:21.227984 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:22:21.227954    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f1f12a_5579_4eb6_be7c_7ee8140abf96.slice/crio-bdd4e723bbf794df4f987b10f1536810fdd81cd2331ff5728e728d6bac733532 WatchSource:0}: Error finding container bdd4e723bbf794df4f987b10f1536810fdd81cd2331ff5728e728d6bac733532: Status 404 returned error can't find the container with id bdd4e723bbf794df4f987b10f1536810fdd81cd2331ff5728e728d6bac733532
Jul 07 13:22:21.241393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.241369    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mv4db"]
Jul 07 13:22:21.245597 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.245577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv4db"
Jul 07 13:22:21.249312 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.249296    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-djfkd\""
Jul 07 13:22:21.259221 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.259200    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv4db"]
Jul 07 13:22:21.345895 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.345845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqvn8\" (UniqueName: \"kubernetes.io/projected/12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a-kube-api-access-vqvn8\") pod \"authorino-58cd955675-mv4db\" (UID: \"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a\") " pod="kuadrant-system/authorino-58cd955675-mv4db"
Jul 07 13:22:21.446710 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.446691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqvn8\" (UniqueName: \"kubernetes.io/projected/12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a-kube-api-access-vqvn8\") pod \"authorino-58cd955675-mv4db\" (UID: \"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a\") " pod="kuadrant-system/authorino-58cd955675-mv4db"
Jul 07 13:22:21.460317 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.460290    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqvn8\" (UniqueName: \"kubernetes.io/projected/12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a-kube-api-access-vqvn8\") pod \"authorino-58cd955675-mv4db\" (UID: \"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a\") " pod="kuadrant-system/authorino-58cd955675-mv4db"
Jul 07 13:22:21.496912 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.496887    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7gs7g"]
Jul 07 13:22:21.499995 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.499980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7gs7g"
Jul 07 13:22:21.515136 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.515112    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7gs7g"]
Jul 07 13:22:21.554911 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.554889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv4db"
Jul 07 13:22:21.620230 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.620203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9" event={"ID":"f3f1f12a-5579-4eb6-be7c-7ee8140abf96","Type":"ContainerStarted","Data":"bdd4e723bbf794df4f987b10f1536810fdd81cd2331ff5728e728d6bac733532"}
Jul 07 13:22:21.648264 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.648233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7nvx\" (UniqueName: \"kubernetes.io/projected/aaad6531-6db4-4145-8aab-2bbbe10c9bd8-kube-api-access-h7nvx\") pod \"authorino-75d556bd77-7gs7g\" (UID: \"aaad6531-6db4-4145-8aab-2bbbe10c9bd8\") " pod="kuadrant-system/authorino-75d556bd77-7gs7g"
Jul 07 13:22:21.681986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.681956    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv4db"]
Jul 07 13:22:21.684617 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:22:21.684589    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12c309b9_a8cf_4b89_a1f7_abb8c59b5d2a.slice/crio-e5908897cdbd43180c727faec95f59787c834aaa942eae54305a16604da26c4d WatchSource:0}: Error finding container e5908897cdbd43180c727faec95f59787c834aaa942eae54305a16604da26c4d: Status 404 returned error can't find the container with id e5908897cdbd43180c727faec95f59787c834aaa942eae54305a16604da26c4d
Jul 07 13:22:21.748845 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.748823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7nvx\" (UniqueName: \"kubernetes.io/projected/aaad6531-6db4-4145-8aab-2bbbe10c9bd8-kube-api-access-h7nvx\") pod \"authorino-75d556bd77-7gs7g\" (UID: \"aaad6531-6db4-4145-8aab-2bbbe10c9bd8\") " pod="kuadrant-system/authorino-75d556bd77-7gs7g"
Jul 07 13:22:21.762950 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.762922    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7nvx\" (UniqueName: \"kubernetes.io/projected/aaad6531-6db4-4145-8aab-2bbbe10c9bd8-kube-api-access-h7nvx\") pod \"authorino-75d556bd77-7gs7g\" (UID: \"aaad6531-6db4-4145-8aab-2bbbe10c9bd8\") " pod="kuadrant-system/authorino-75d556bd77-7gs7g"
Jul 07 13:22:21.808712 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.808686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7gs7g"
Jul 07 13:22:21.976922 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:21.976889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7gs7g"]
Jul 07 13:22:21.982080 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:22:21.982049    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaaad6531_6db4_4145_8aab_2bbbe10c9bd8.slice/crio-28b88828fd31e9f8f9b161f4226e269d7c86e3aef1f49335dabd48fd3952be47 WatchSource:0}: Error finding container 28b88828fd31e9f8f9b161f4226e269d7c86e3aef1f49335dabd48fd3952be47: Status 404 returned error can't find the container with id 28b88828fd31e9f8f9b161f4226e269d7c86e3aef1f49335dabd48fd3952be47
Jul 07 13:22:22.637541 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:22.637452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv4db" event={"ID":"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a","Type":"ContainerStarted","Data":"e5908897cdbd43180c727faec95f59787c834aaa942eae54305a16604da26c4d"}
Jul 07 13:22:22.640106 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:22.640074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7gs7g" event={"ID":"aaad6531-6db4-4145-8aab-2bbbe10c9bd8","Type":"ContainerStarted","Data":"28b88828fd31e9f8f9b161f4226e269d7c86e3aef1f49335dabd48fd3952be47"}
Jul 07 13:22:26.665657 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.665622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv4db" event={"ID":"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a","Type":"ContainerStarted","Data":"ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e"}
Jul 07 13:22:26.667057 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.667033    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7gs7g" event={"ID":"aaad6531-6db4-4145-8aab-2bbbe10c9bd8","Type":"ContainerStarted","Data":"2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28"}
Jul 07 13:22:26.668389 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.668370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9" event={"ID":"f3f1f12a-5579-4eb6-be7c-7ee8140abf96","Type":"ContainerStarted","Data":"103c3c40f3b4a653a275df521dfbc87fa18b608efc2516cabbca7621b75b5e2e"}
Jul 07 13:22:26.668511 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.668436    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:26.684847 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.684782    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mv4db" podStartSLOduration=1.185656143 podStartE2EDuration="5.684770442s" totalImagesPullingTime="4.499114299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:26.68305914 +0000 UTC m=+531.700215336" watchObservedRunningTime="2026-07-07 13:22:26.684770442 +0000 UTC m=+531.701926625"
Jul 07 13:22:26.703785 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.703732    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9" podStartSLOduration=1.70803196 podStartE2EDuration="6.703716714s" totalImagesPullingTime="4.995684754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:26.702513425 +0000 UTC m=+531.719669640" watchObservedRunningTime="2026-07-07 13:22:26.703716714 +0000 UTC m=+531.720872899"
Jul 07 13:22:26.722518 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.722480    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7gs7g" podStartSLOduration=1.533287449 podStartE2EDuration="5.722468928s" totalImagesPullingTime="4.189181479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:26.721488864 +0000 UTC m=+531.738645046" watchObservedRunningTime="2026-07-07 13:22:26.722468928 +0000 UTC m=+531.739625111"
Jul 07 13:22:26.766582 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:26.766560    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv4db"]
Jul 07 13:22:28.676199 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:28.676120    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mv4db" podUID="12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a" containerName="authorino" containerID="cri-o://ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e" gracePeriod=30
Jul 07 13:22:28.924439 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:28.924412    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv4db"
Jul 07 13:22:28.997331 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:28.997311    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqvn8\" (UniqueName: \"kubernetes.io/projected/12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a-kube-api-access-vqvn8\") pod \"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a\" (UID: \"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a\") "
Jul 07 13:22:28.999351 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:28.999325    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a-kube-api-access-vqvn8" (OuterVolumeSpecName: "kube-api-access-vqvn8") pod "12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a" (UID: "12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a"). InnerVolumeSpecName "kube-api-access-vqvn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:22:29.097756 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.097732    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqvn8\" (UniqueName: \"kubernetes.io/projected/12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a-kube-api-access-vqvn8\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:22:29.681577 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.681545    2577 generic.go:356] "Generic (PLEG): container finished" podID="12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a" containerID="ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e" exitCode=0
Jul 07 13:22:29.682001 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.681594    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mv4db"
Jul 07 13:22:29.682001 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.681629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv4db" event={"ID":"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a","Type":"ContainerDied","Data":"ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e"}
Jul 07 13:22:29.682001 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.681668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mv4db" event={"ID":"12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a","Type":"ContainerDied","Data":"e5908897cdbd43180c727faec95f59787c834aaa942eae54305a16604da26c4d"}
Jul 07 13:22:29.682001 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.681686    2577 scope.go:117] "RemoveContainer" containerID="ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e"
Jul 07 13:22:29.689773 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.689754    2577 scope.go:117] "RemoveContainer" containerID="ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e"
Jul 07 13:22:29.690014 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:22:29.689998    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e\": container with ID starting with ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e not found: ID does not exist" containerID="ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e"
Jul 07 13:22:29.690073 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.690022    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e"} err="failed to get container status \"ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e\": rpc error: code = NotFound desc = could not find container \"ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e\": container with ID starting with ef3d2d1e4e3a3008e7ba2da4b453036ad35c975ddef6a8e5c2d362346405e98e not found: ID does not exist"
Jul 07 13:22:29.706266 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.706241    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv4db"]
Jul 07 13:22:29.708400 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:29.708378    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mv4db"]
Jul 07 13:22:31.513714 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:31.513674    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a" path="/var/lib/kubelet/pods/12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a/volumes"
Jul 07 13:22:37.673014 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:37.672988    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"
Jul 07 13:22:54.003993 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.003955    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j25v5"]
Jul 07 13:22:54.004549 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.004402    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a" containerName="authorino"
Jul 07 13:22:54.004549 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.004422    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a" containerName="authorino"
Jul 07 13:22:54.004549 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.004501    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c309b9-a8cf-4b89-a1f7-abb8c59b5d2a" containerName="authorino"
Jul 07 13:22:54.008718 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.008697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j25v5"
Jul 07 13:22:54.017107 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.017087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j25v5"]
Jul 07 13:22:54.079871 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.079850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p56fg\" (UniqueName: \"kubernetes.io/projected/4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee-kube-api-access-p56fg\") pod \"authorino-5d4b67c446-j25v5\" (UID: \"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee\") " pod="kuadrant-system/authorino-5d4b67c446-j25v5"
Jul 07 13:22:54.180733 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.180709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p56fg\" (UniqueName: \"kubernetes.io/projected/4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee-kube-api-access-p56fg\") pod \"authorino-5d4b67c446-j25v5\" (UID: \"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee\") " pod="kuadrant-system/authorino-5d4b67c446-j25v5"
Jul 07 13:22:54.190245 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.190228    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p56fg\" (UniqueName: \"kubernetes.io/projected/4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee-kube-api-access-p56fg\") pod \"authorino-5d4b67c446-j25v5\" (UID: \"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee\") " pod="kuadrant-system/authorino-5d4b67c446-j25v5"
Jul 07 13:22:54.250234 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.250212    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j25v5"]
Jul 07 13:22:54.250430 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.250405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j25v5"
Jul 07 13:22:54.286349 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.286324    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69fd9fbcc4-2kj4p"]
Jul 07 13:22:54.289798 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.289776    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p"
Jul 07 13:22:54.300672 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.300638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69fd9fbcc4-2kj4p"]
Jul 07 13:22:54.381793 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.381761    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j25v5"]
Jul 07 13:22:54.382115 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.382090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zjx9\" (UniqueName: \"kubernetes.io/projected/42118b89-aa9e-48d2-9235-8d0c762d63cd-kube-api-access-6zjx9\") pod \"authorino-69fd9fbcc4-2kj4p\" (UID: \"42118b89-aa9e-48d2-9235-8d0c762d63cd\") " pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p"
Jul 07 13:22:54.384582 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:22:54.384553    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b530976_6b7d_41ce_bdcc_dcb3ce65d7ee.slice/crio-0c655950747e8abaa66153221bd2a0a1dd5530036d55899a40445aed7680794b WatchSource:0}: Error finding container 0c655950747e8abaa66153221bd2a0a1dd5530036d55899a40445aed7680794b: Status 404 returned error can't find the container with id 0c655950747e8abaa66153221bd2a0a1dd5530036d55899a40445aed7680794b
Jul 07 13:22:54.448886 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.448860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69fd9fbcc4-2kj4p"]
Jul 07 13:22:54.449105 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:22:54.449083    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-6zjx9], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p" podUID="42118b89-aa9e-48d2-9235-8d0c762d63cd"
Jul 07 13:22:54.483096 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.483069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zjx9\" (UniqueName: \"kubernetes.io/projected/42118b89-aa9e-48d2-9235-8d0c762d63cd-kube-api-access-6zjx9\") pod \"authorino-69fd9fbcc4-2kj4p\" (UID: \"42118b89-aa9e-48d2-9235-8d0c762d63cd\") " pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p"
Jul 07 13:22:54.483446 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.483426    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-586c79cf86-n8pwt"]
Jul 07 13:22:54.486764 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.486747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.490260 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.490246    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 13:22:54.496851 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.496829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-586c79cf86-n8pwt"]
Jul 07 13:22:54.501827 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.501792    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zjx9\" (UniqueName: \"kubernetes.io/projected/42118b89-aa9e-48d2-9235-8d0c762d63cd-kube-api-access-6zjx9\") pod \"authorino-69fd9fbcc4-2kj4p\" (UID: \"42118b89-aa9e-48d2-9235-8d0c762d63cd\") " pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p"
Jul 07 13:22:54.584303 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.584249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8bkf\" (UniqueName: \"kubernetes.io/projected/642a057e-d379-4531-96a7-284bf508f08c-kube-api-access-v8bkf\") pod \"authorino-586c79cf86-n8pwt\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") " pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.584378 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.584317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a057e-d379-4531-96a7-284bf508f08c-tls-cert\") pod \"authorino-586c79cf86-n8pwt\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") " pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.685697 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.685675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8bkf\" (UniqueName: \"kubernetes.io/projected/642a057e-d379-4531-96a7-284bf508f08c-kube-api-access-v8bkf\") pod \"authorino-586c79cf86-n8pwt\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") " pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.685804 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.685716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a057e-d379-4531-96a7-284bf508f08c-tls-cert\") pod \"authorino-586c79cf86-n8pwt\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") " pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.687802 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.687779    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a057e-d379-4531-96a7-284bf508f08c-tls-cert\") pod \"authorino-586c79cf86-n8pwt\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") " pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.696476 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.696455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8bkf\" (UniqueName: \"kubernetes.io/projected/642a057e-d379-4531-96a7-284bf508f08c-kube-api-access-v8bkf\") pod \"authorino-586c79cf86-n8pwt\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") " pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.781892 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.781869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j25v5" event={"ID":"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee","Type":"ContainerStarted","Data":"0c655950747e8abaa66153221bd2a0a1dd5530036d55899a40445aed7680794b"}
Jul 07 13:22:54.781994 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.781929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p"
Jul 07 13:22:54.789617 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.789598    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p"
Jul 07 13:22:54.796624 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.796607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:22:54.886492 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.886463    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zjx9\" (UniqueName: \"kubernetes.io/projected/42118b89-aa9e-48d2-9235-8d0c762d63cd-kube-api-access-6zjx9\") pod \"42118b89-aa9e-48d2-9235-8d0c762d63cd\" (UID: \"42118b89-aa9e-48d2-9235-8d0c762d63cd\") "
Jul 07 13:22:54.889882 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.889847    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42118b89-aa9e-48d2-9235-8d0c762d63cd-kube-api-access-6zjx9" (OuterVolumeSpecName: "kube-api-access-6zjx9") pod "42118b89-aa9e-48d2-9235-8d0c762d63cd" (UID: "42118b89-aa9e-48d2-9235-8d0c762d63cd"). InnerVolumeSpecName "kube-api-access-6zjx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:22:54.941308 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.941280    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-586c79cf86-n8pwt"]
Jul 07 13:22:54.943618 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:22:54.943593    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod642a057e_d379_4531_96a7_284bf508f08c.slice/crio-9d9fd344c0bbf0d6d0080fb21f706ceb2413ee16632579b56bb56074e1318709 WatchSource:0}: Error finding container 9d9fd344c0bbf0d6d0080fb21f706ceb2413ee16632579b56bb56074e1318709: Status 404 returned error can't find the container with id 9d9fd344c0bbf0d6d0080fb21f706ceb2413ee16632579b56bb56074e1318709
Jul 07 13:22:54.987235 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:54.987211    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zjx9\" (UniqueName: \"kubernetes.io/projected/42118b89-aa9e-48d2-9235-8d0c762d63cd-kube-api-access-6zjx9\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:22:55.786684 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.786596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j25v5" event={"ID":"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee","Type":"ContainerStarted","Data":"7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c"}
Jul 07 13:22:55.786684 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.786635    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-j25v5" podUID="4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee" containerName="authorino" containerID="cri-o://7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c" gracePeriod=30
Jul 07 13:22:55.788019 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.787993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586c79cf86-n8pwt" event={"ID":"642a057e-d379-4531-96a7-284bf508f08c","Type":"ContainerStarted","Data":"038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac"}
Jul 07 13:22:55.788123 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.788033    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586c79cf86-n8pwt" event={"ID":"642a057e-d379-4531-96a7-284bf508f08c","Type":"ContainerStarted","Data":"9d9fd344c0bbf0d6d0080fb21f706ceb2413ee16632579b56bb56074e1318709"}
Jul 07 13:22:55.788123 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.788001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69fd9fbcc4-2kj4p"
Jul 07 13:22:55.841102 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.841063    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-j25v5" podStartSLOduration=2.477562136 podStartE2EDuration="2.841048917s" totalImagesPullingTime="363.486781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:55.81185823 +0000 UTC m=+560.829014412" watchObservedRunningTime="2026-07-07 13:22:55.841048917 +0000 UTC m=+560.858205114"
Jul 07 13:22:55.858489 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.858466    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69fd9fbcc4-2kj4p"]
Jul 07 13:22:55.862050 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.862028    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69fd9fbcc4-2kj4p"]
Jul 07 13:22:55.886667 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.886627    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-586c79cf86-n8pwt" podStartSLOduration=1.584796015 podStartE2EDuration="1.886615294s" totalImagesPullingTime="301.819279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:55.885691417 +0000 UTC m=+560.902847583" watchObservedRunningTime="2026-07-07 13:22:55.886615294 +0000 UTC m=+560.903771477"
Jul 07 13:22:55.922318 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.922287    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7gs7g"]
Jul 07 13:22:55.922479 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:55.922459    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7gs7g" podUID="aaad6531-6db4-4145-8aab-2bbbe10c9bd8" containerName="authorino" containerID="cri-o://2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28" gracePeriod=30
Jul 07 13:22:56.036930 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.036878    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j25v5"
Jul 07 13:22:56.094260 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.094234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p56fg\" (UniqueName: \"kubernetes.io/projected/4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee-kube-api-access-p56fg\") pod \"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee\" (UID: \"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee\") "
Jul 07 13:22:56.099103 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.099074    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee-kube-api-access-p56fg" (OuterVolumeSpecName: "kube-api-access-p56fg") pod "4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee" (UID: "4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee"). InnerVolumeSpecName "kube-api-access-p56fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:22:56.150312 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.150290    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7gs7g"
Jul 07 13:22:56.195597 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.195565    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7nvx\" (UniqueName: \"kubernetes.io/projected/aaad6531-6db4-4145-8aab-2bbbe10c9bd8-kube-api-access-h7nvx\") pod \"aaad6531-6db4-4145-8aab-2bbbe10c9bd8\" (UID: \"aaad6531-6db4-4145-8aab-2bbbe10c9bd8\") "
Jul 07 13:22:56.195794 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.195778    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p56fg\" (UniqueName: \"kubernetes.io/projected/4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee-kube-api-access-p56fg\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:22:56.197847 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.197806    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaad6531-6db4-4145-8aab-2bbbe10c9bd8-kube-api-access-h7nvx" (OuterVolumeSpecName: "kube-api-access-h7nvx") pod "aaad6531-6db4-4145-8aab-2bbbe10c9bd8" (UID: "aaad6531-6db4-4145-8aab-2bbbe10c9bd8"). InnerVolumeSpecName "kube-api-access-h7nvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:22:56.296425 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.296374    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7nvx\" (UniqueName: \"kubernetes.io/projected/aaad6531-6db4-4145-8aab-2bbbe10c9bd8-kube-api-access-h7nvx\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:22:56.793703 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.793671    2577 generic.go:356] "Generic (PLEG): container finished" podID="aaad6531-6db4-4145-8aab-2bbbe10c9bd8" containerID="2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28" exitCode=0
Jul 07 13:22:56.794121 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.793717    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7gs7g"
Jul 07 13:22:56.794121 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.793753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7gs7g" event={"ID":"aaad6531-6db4-4145-8aab-2bbbe10c9bd8","Type":"ContainerDied","Data":"2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28"}
Jul 07 13:22:56.794121 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.793789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7gs7g" event={"ID":"aaad6531-6db4-4145-8aab-2bbbe10c9bd8","Type":"ContainerDied","Data":"28b88828fd31e9f8f9b161f4226e269d7c86e3aef1f49335dabd48fd3952be47"}
Jul 07 13:22:56.794121 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.793823    2577 scope.go:117] "RemoveContainer" containerID="2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28"
Jul 07 13:22:56.795202 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.795182    2577 generic.go:356] "Generic (PLEG): container finished" podID="4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee" containerID="7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c" exitCode=0
Jul 07 13:22:56.795292 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.795240    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j25v5"
Jul 07 13:22:56.795292 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.795275    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j25v5" event={"ID":"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee","Type":"ContainerDied","Data":"7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c"}
Jul 07 13:22:56.795393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.795300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j25v5" event={"ID":"4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee","Type":"ContainerDied","Data":"0c655950747e8abaa66153221bd2a0a1dd5530036d55899a40445aed7680794b"}
Jul 07 13:22:56.803006 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.802991    2577 scope.go:117] "RemoveContainer" containerID="2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28"
Jul 07 13:22:56.803263 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:22:56.803238    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28\": container with ID starting with 2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28 not found: ID does not exist" containerID="2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28"
Jul 07 13:22:56.803365 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.803268    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28"} err="failed to get container status \"2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28\": rpc error: code = NotFound desc = could not find container \"2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28\": container with ID starting with 2adb5d57b654bfc5a10dd51318ce1e9fc72736e8a5f56b43cb5e50403daa2c28 not found: ID does not exist"
Jul 07 13:22:56.803365 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.803284    2577 scope.go:117] "RemoveContainer" containerID="7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c"
Jul 07 13:22:56.810737 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.810719    2577 scope.go:117] "RemoveContainer" containerID="7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c"
Jul 07 13:22:56.810988 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:22:56.810970    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c\": container with ID starting with 7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c not found: ID does not exist" containerID="7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c"
Jul 07 13:22:56.811051 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.810993    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c"} err="failed to get container status \"7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c\": rpc error: code = NotFound desc = could not find container \"7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c\": container with ID starting with 7bfbf394a265eaa59977c4922fb4e23b3d1f1353d8888742100033ce53682a6c not found: ID does not exist"
Jul 07 13:22:56.853325 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.853304    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7gs7g"]
Jul 07 13:22:56.864529 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.864480    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7gs7g"]
Jul 07 13:22:56.888248 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.888225    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j25v5"]
Jul 07 13:22:56.895635 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:56.895614    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j25v5"]
Jul 07 13:22:57.514917 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:57.514887    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42118b89-aa9e-48d2-9235-8d0c762d63cd" path="/var/lib/kubelet/pods/42118b89-aa9e-48d2-9235-8d0c762d63cd/volumes"
Jul 07 13:22:57.515103 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:57.515091    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee" path="/var/lib/kubelet/pods/4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee/volumes"
Jul 07 13:22:57.515372 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:22:57.515359    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaad6531-6db4-4145-8aab-2bbbe10c9bd8" path="/var/lib/kubelet/pods/aaad6531-6db4-4145-8aab-2bbbe10c9bd8/volumes"
Jul 07 13:23:35.423970 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:23:35.423939    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:23:35.428644 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:23:35.428625    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:23:37.833765 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:23:37.833729    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:23:55.776480 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:23:55.776437    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:24:02.180482 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:02.180450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:24:13.989384 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:13.989349    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:24:30.228535 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:30.228494    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:24:39.489413 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:39.489377    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:24:44.415032 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.414998    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"]
Jul 07 13:24:44.415389 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.415289    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaad6531-6db4-4145-8aab-2bbbe10c9bd8" containerName="authorino"
Jul 07 13:24:44.415389 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.415302    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaad6531-6db4-4145-8aab-2bbbe10c9bd8" containerName="authorino"
Jul 07 13:24:44.415389 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.415317    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee" containerName="authorino"
Jul 07 13:24:44.415389 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.415322    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee" containerName="authorino"
Jul 07 13:24:44.415389 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.415379    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaad6531-6db4-4145-8aab-2bbbe10c9bd8" containerName="authorino"
Jul 07 13:24:44.415389 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.415390    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b530976-6b7d-41ce-bdcc-dcb3ce65d7ee" containerName="authorino"
Jul 07 13:24:44.418329 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.418311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.422440 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.422411    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 13:24:44.422440 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.422424    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 13:24:44.423967 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.423947    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 13:24:44.424067 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.423974    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jn6x5\""
Jul 07 13:24:44.431734 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.431714    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"]
Jul 07 13:24:44.491024 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.490999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.491188 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.491068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.491188 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.491097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wqvf\" (UniqueName: \"kubernetes.io/projected/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-kube-api-access-2wqvf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.491188 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.491133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.491346 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.491189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.491346 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.491272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592387 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wqvf\" (UniqueName: \"kubernetes.io/projected/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-kube-api-access-2wqvf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592860 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592835    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592940 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592857    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.592940 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.592908    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.594742 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.594723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.595054 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.595033    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.602213 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.602194    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wqvf\" (UniqueName: \"kubernetes.io/projected/d9c315bd-9f27-4cdb-b0ed-fc27af79979b-kube-api-access-2wqvf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr\" (UID: \"d9c315bd-9f27-4cdb-b0ed-fc27af79979b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.726991 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.726921    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:44.867484 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.867456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"]
Jul 07 13:24:44.869475 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:24:44.869448    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9c315bd_9f27_4cdb_b0ed_fc27af79979b.slice/crio-66b209afe5cf3f211b189c71cd89572893a863e2d123ce72e38ec312a3ccb6ab WatchSource:0}: Error finding container 66b209afe5cf3f211b189c71cd89572893a863e2d123ce72e38ec312a3ccb6ab: Status 404 returned error can't find the container with id 66b209afe5cf3f211b189c71cd89572893a863e2d123ce72e38ec312a3ccb6ab
Jul 07 13:24:44.870764 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:44.870746    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:24:45.254713 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:45.254662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr" event={"ID":"d9c315bd-9f27-4cdb-b0ed-fc27af79979b","Type":"ContainerStarted","Data":"66b209afe5cf3f211b189c71cd89572893a863e2d123ce72e38ec312a3ccb6ab"}
Jul 07 13:24:46.260807 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:46.260774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr" event={"ID":"d9c315bd-9f27-4cdb-b0ed-fc27af79979b","Type":"ContainerStarted","Data":"a425b256030d8ad6d4d125fd7cfe8730272dae3b6036c85272b346b0c6b751eb"}
Jul 07 13:24:46.306096 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:46.306051    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr" podStartSLOduration=1.261883701 podStartE2EDuration="2.306036844s" totalImagesPullingTime="1.044153143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:24:46.304345601 +0000 UTC m=+671.321501784" watchObservedRunningTime="2026-07-07 13:24:46.306036844 +0000 UTC m=+671.323193026"
Jul 07 13:24:48.478917 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:48.478874    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:24:54.727040 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:54.727005    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:54.727379 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:54.727062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:54.739193 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:54.739173    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:24:55.311462 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:24:55.311431    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr"
Jul 07 13:25:51.152727 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.152694    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c784fc7ff-486df"]
Jul 07 13:25:51.155884 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.155867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.166743 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.166719    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c784fc7ff-486df"]
Jul 07 13:25:51.273751 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.273726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jx6q\" (UniqueName: \"kubernetes.io/projected/f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b-kube-api-access-6jx6q\") pod \"authorino-c784fc7ff-486df\" (UID: \"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b\") " pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.273900 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.273759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b-tls-cert\") pod \"authorino-c784fc7ff-486df\" (UID: \"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b\") " pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.374252 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.374226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jx6q\" (UniqueName: \"kubernetes.io/projected/f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b-kube-api-access-6jx6q\") pod \"authorino-c784fc7ff-486df\" (UID: \"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b\") " pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.374366 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.374267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b-tls-cert\") pod \"authorino-c784fc7ff-486df\" (UID: \"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b\") " pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.376687 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.376669    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b-tls-cert\") pod \"authorino-c784fc7ff-486df\" (UID: \"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b\") " pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.382961 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.382944    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jx6q\" (UniqueName: \"kubernetes.io/projected/f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b-kube-api-access-6jx6q\") pod \"authorino-c784fc7ff-486df\" (UID: \"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b\") " pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.463865 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.463801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c784fc7ff-486df"
Jul 07 13:25:51.593127 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:51.593099    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c784fc7ff-486df"]
Jul 07 13:25:51.596622 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:25:51.596586    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3e38c1c_dc58_4836_8b19_a6ed4edd2f0b.slice/crio-fef42d0b3034662f857951af216e9cbdafb0f95865cfcd79f9a774728cbaf9c4 WatchSource:0}: Error finding container fef42d0b3034662f857951af216e9cbdafb0f95865cfcd79f9a774728cbaf9c4: Status 404 returned error can't find the container with id fef42d0b3034662f857951af216e9cbdafb0f95865cfcd79f9a774728cbaf9c4
Jul 07 13:25:52.531666 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:52.531634    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c784fc7ff-486df" event={"ID":"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b","Type":"ContainerStarted","Data":"fef42d0b3034662f857951af216e9cbdafb0f95865cfcd79f9a774728cbaf9c4"}
Jul 07 13:25:53.537363 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.537325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c784fc7ff-486df" event={"ID":"f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b","Type":"ContainerStarted","Data":"91a435c2cd28408c3c191b5ed39ebd9fb082ffac375fdfd4dc52bffa15de121e"}
Jul 07 13:25:53.559418 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.559357    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c784fc7ff-486df" podStartSLOduration=1.621084778 podStartE2EDuration="2.559337442s" totalImagesPullingTime="938.252664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:25:53.558154649 +0000 UTC m=+738.575310833" watchObservedRunningTime="2026-07-07 13:25:53.559337442 +0000 UTC m=+738.576493627"
Jul 07 13:25:53.594161 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.594129    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-586c79cf86-n8pwt"]
Jul 07 13:25:53.594356 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.594328    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-586c79cf86-n8pwt" podUID="642a057e-d379-4531-96a7-284bf508f08c" containerName="authorino" containerID="cri-o://038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac" gracePeriod=30
Jul 07 13:25:53.841913 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.841886    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:25:53.895319 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.895292    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a057e-d379-4531-96a7-284bf508f08c-tls-cert\") pod \"642a057e-d379-4531-96a7-284bf508f08c\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") "
Jul 07 13:25:53.895485 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.895352    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8bkf\" (UniqueName: \"kubernetes.io/projected/642a057e-d379-4531-96a7-284bf508f08c-kube-api-access-v8bkf\") pod \"642a057e-d379-4531-96a7-284bf508f08c\" (UID: \"642a057e-d379-4531-96a7-284bf508f08c\") "
Jul 07 13:25:53.897464 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.897431    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642a057e-d379-4531-96a7-284bf508f08c-kube-api-access-v8bkf" (OuterVolumeSpecName: "kube-api-access-v8bkf") pod "642a057e-d379-4531-96a7-284bf508f08c" (UID: "642a057e-d379-4531-96a7-284bf508f08c"). InnerVolumeSpecName "kube-api-access-v8bkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:25:53.904986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.904955    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/642a057e-d379-4531-96a7-284bf508f08c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "642a057e-d379-4531-96a7-284bf508f08c" (UID: "642a057e-d379-4531-96a7-284bf508f08c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:25:53.996193 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.996153    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a057e-d379-4531-96a7-284bf508f08c-tls-cert\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:25:53.996193 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:53.996188    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8bkf\" (UniqueName: \"kubernetes.io/projected/642a057e-d379-4531-96a7-284bf508f08c-kube-api-access-v8bkf\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:25:54.543554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.543513    2577 generic.go:356] "Generic (PLEG): container finished" podID="642a057e-d379-4531-96a7-284bf508f08c" containerID="038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac" exitCode=0
Jul 07 13:25:54.544022 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.543573    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586c79cf86-n8pwt"
Jul 07 13:25:54.544022 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.543612    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586c79cf86-n8pwt" event={"ID":"642a057e-d379-4531-96a7-284bf508f08c","Type":"ContainerDied","Data":"038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac"}
Jul 07 13:25:54.544022 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.543662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586c79cf86-n8pwt" event={"ID":"642a057e-d379-4531-96a7-284bf508f08c","Type":"ContainerDied","Data":"9d9fd344c0bbf0d6d0080fb21f706ceb2413ee16632579b56bb56074e1318709"}
Jul 07 13:25:54.544022 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.543685    2577 scope.go:117] "RemoveContainer" containerID="038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac"
Jul 07 13:25:54.551912 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.551893    2577 scope.go:117] "RemoveContainer" containerID="038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac"
Jul 07 13:25:54.552143 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:25:54.552125    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac\": container with ID starting with 038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac not found: ID does not exist" containerID="038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac"
Jul 07 13:25:54.552193 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.552151    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac"} err="failed to get container status \"038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac\": rpc error: code = NotFound desc = could not find container \"038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac\": container with ID starting with 038d0560629efa97c8d3e52f5e66385f021872883995b9bba692d27354c133ac not found: ID does not exist"
Jul 07 13:25:54.568227 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.568202    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-586c79cf86-n8pwt"]
Jul 07 13:25:54.573223 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:54.573199    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-586c79cf86-n8pwt"]
Jul 07 13:25:55.513159 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:25:55.513129    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="642a057e-d379-4531-96a7-284bf508f08c" path="/var/lib/kubelet/pods/642a057e-d379-4531-96a7-284bf508f08c/volumes"
Jul 07 13:26:22.292028 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:26:22.291996    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:26:31.374805 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:26:31.374772    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:26:44.077917 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:26:44.077885    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:26:51.879702 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:26:51.879667    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:27:02.784959 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:27:02.784928    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:27:28.382332 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:27:28.382301    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:27:39.983650 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:27:39.983617    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:28:14.778422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:28:14.778392    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:28:24.879634 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:28:24.879601    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:28:35.391236 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:28:35.391202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:28:35.450278 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:28:35.450247    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:28:35.452292 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:28:35.452269    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:28:43.777100 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:28:43.777067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:28:54.387642 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:28:54.387597    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:29:03.684874 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:29:03.684844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:29:14.283657 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:29:14.283623    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:29:23.988107 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:29:23.988074    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:29:46.385912 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:29:46.385876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:29:59.375860 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:29:59.375826    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:30:23.182376 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:30:23.182343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:30:34.196990 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:30:34.196956    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:30:49.598985 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:30:49.598949    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:31:00.387421 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:31:00.387388    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:31:10.377824 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:31:10.377771    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:31:19.382562 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:31:19.382527    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:31:28.180795 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:31:28.180695    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:33:17.594642 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:33:17.594608    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:33:26.987337 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:33:26.987304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:33:35.475472 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:33:35.475440    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:33:35.476069 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:33:35.476041    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:33:40.088295 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:33:40.088256    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:33:50.696338 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:33:50.696302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:34:03.385671 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:34:03.385634    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:34:12.806564 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:34:12.806531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:34:21.992824 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:34:21.992771    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:34:32.879958 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:34:32.879881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:34:44.780835 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:34:44.780782    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:34:53.279886 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:34:53.279851    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:35:03.490464 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:35:03.490429    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:35:25.891357 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:35:25.891324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:35:35.086873 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:35:35.086836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:35:45.286157 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:35:45.286118    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:35:56.087027 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:35:56.086989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:36:13.201966 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:36:13.201922    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:36:24.977709 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:36:24.977676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:36:34.385847 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:36:34.385799    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:36:44.492141 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:36:44.492107    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:36:53.682475 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:36:53.682434    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:37:02.685544 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:37:02.685503    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:37:15.782197 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:37:15.782165    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:37:25.076507 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:37:25.076474    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:37:34.492073 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:37:34.491993    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:37:46.584144 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:37:46.584104    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:37:57.578314 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:37:57.578277    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:38:05.285862 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:38:05.285828    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:38:17.592191 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:38:17.592156    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:38:29.984325 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:38:29.984289    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:38:35.500327 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:38:35.500277    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:38:35.501270 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:38:35.501248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:38:42.997883 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:38:42.997853    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:38:52.276828 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:38:52.276781    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:39:04.083335 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:39:04.083300    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:39:12.686213 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:39:12.686177    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:39:24.883402 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:39:24.883368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:39:31.555746 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:39:31.555672    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:39:53.983779 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:39:53.983744    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:40:02.191759 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:40:02.191723    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:41:25.132741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:41:25.132706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:41:34.085996 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:41:34.084229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:41:47.104589 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:41:47.104554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:41:57.784134 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:41:57.784099    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:42:29.599393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:29.599360    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:42:46.500698 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.500668    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"]
Jul 07 13:42:46.501172 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.501020    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="642a057e-d379-4531-96a7-284bf508f08c" containerName="authorino"
Jul 07 13:42:46.501172 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.501038    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="642a057e-d379-4531-96a7-284bf508f08c" containerName="authorino"
Jul 07 13:42:46.501172 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.501154    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="642a057e-d379-4531-96a7-284bf508f08c" containerName="authorino"
Jul 07 13:42:46.504094 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.504073    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.507136 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.507111    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8880b7f7\"/\"kube-root-ca.crt\""
Jul 07 13:42:46.508424 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.508404    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8880b7f7\"/\"e2e-lifecycle-model-8880b7f7-kserve-self-signed-certs\""
Jul 07 13:42:46.508528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.508431    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8880b7f7\"/\"default-dockercfg-xvfp7\""
Jul 07 13:42:46.508528 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.508509    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8880b7f7\"/\"openshift-service-ca.crt\""
Jul 07 13:42:46.514449 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.514430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"]
Jul 07 13:42:46.606529 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.606500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zgs7\" (UniqueName: \"kubernetes.io/projected/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-kube-api-access-7zgs7\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.606638 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.606547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-home\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.606638 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.606565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-model-cache\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.606638 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.606627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tls-certs\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.606783 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.606685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-dshm\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.606783 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.606706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tmp-dir\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708002 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.707976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zgs7\" (UniqueName: \"kubernetes.io/projected/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-kube-api-access-7zgs7\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708102 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-home\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708102 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-model-cache\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708237 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tls-certs\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708286 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-dshm\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708286 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tmp-dir\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708359 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708306    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-model-cache\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708400 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708372    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-home\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.708615 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.708596    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tmp-dir\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.710284 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.710260    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-dshm\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.710702 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.710680    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tls-certs\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.717520 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.717498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zgs7\" (UniqueName: \"kubernetes.io/projected/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-kube-api-access-7zgs7\") pod \"e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") " pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.814590 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.814534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:46.947944 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:46.947919    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"]
Jul 07 13:42:46.950232 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:42:46.950202    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e4a2e6f_b328_492a_af9a_3fa580bd54d7.slice/crio-dc6bedc3068fc06dfb80d34d17c83c1480f5ad0e6aa21e566f570c3a40361906 WatchSource:0}: Error finding container dc6bedc3068fc06dfb80d34d17c83c1480f5ad0e6aa21e566f570c3a40361906: Status 404 returned error can't find the container with id dc6bedc3068fc06dfb80d34d17c83c1480f5ad0e6aa21e566f570c3a40361906
Jul 07 13:42:47.740643 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:47.740607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh" event={"ID":"2e4a2e6f-b328-492a-af9a-3fa580bd54d7","Type":"ContainerStarted","Data":"1b35bd5ff246092fcf5667429adbcf295a7c25d74aac19b6d565a67b425f5c27"}
Jul 07 13:42:47.741005 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:47.740652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh" event={"ID":"2e4a2e6f-b328-492a-af9a-3fa580bd54d7","Type":"ContainerStarted","Data":"dc6bedc3068fc06dfb80d34d17c83c1480f5ad0e6aa21e566f570c3a40361906"}
Jul 07 13:42:47.762375 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:47.762324    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh" podStartSLOduration=1.762307414 podStartE2EDuration="1.762307414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:42:47.762200505 +0000 UTC m=+1752.779356688" watchObservedRunningTime="2026-07-07 13:42:47.762307414 +0000 UTC m=+1752.779463598"
Jul 07 13:42:49.515463 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:49.515431    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:42:56.814879 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:56.814848    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:56.815271 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:56.814892    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:56.827600 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:56.827572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:42:57.793936 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:42:57.793904    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:43:04.286494 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:04.286457    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:43:17.484823 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:17.484777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:43:17.769492 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:17.769468    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"]
Jul 07 13:43:32.790625 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:32.790561    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh" podUID="2e4a2e6f-b328-492a-af9a-3fa580bd54d7" containerName="main" containerID="cri-o://1b35bd5ff246092fcf5667429adbcf295a7c25d74aac19b6d565a67b425f5c27" gracePeriod=45
Jul 07 13:43:32.928241 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:32.928206    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e4a2e6f-b328-492a-af9a-3fa580bd54d7" containerID="1b35bd5ff246092fcf5667429adbcf295a7c25d74aac19b6d565a67b425f5c27" exitCode=0
Jul 07 13:43:32.928367 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:32.928236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh" event={"ID":"2e4a2e6f-b328-492a-af9a-3fa580bd54d7","Type":"ContainerDied","Data":"1b35bd5ff246092fcf5667429adbcf295a7c25d74aac19b6d565a67b425f5c27"}
Jul 07 13:43:33.032747 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.032727    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:43:33.141241 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141171    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-home\") pod \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") "
Jul 07 13:43:33.141241 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141214    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tls-certs\") pod \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") "
Jul 07 13:43:33.141241 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141239    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-dshm\") pod \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") "
Jul 07 13:43:33.141480 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141267    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tmp-dir\") pod \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") "
Jul 07 13:43:33.141480 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141308    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zgs7\" (UniqueName: \"kubernetes.io/projected/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-kube-api-access-7zgs7\") pod \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") "
Jul 07 13:43:33.141480 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-model-cache\") pod \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\" (UID: \"2e4a2e6f-b328-492a-af9a-3fa580bd54d7\") "
Jul 07 13:43:33.141480 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141448    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-home" (OuterVolumeSpecName: "home") pod "2e4a2e6f-b328-492a-af9a-3fa580bd54d7" (UID: "2e4a2e6f-b328-492a-af9a-3fa580bd54d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:43:33.141675 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141575    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e4a2e6f-b328-492a-af9a-3fa580bd54d7" (UID: "2e4a2e6f-b328-492a-af9a-3fa580bd54d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:43:33.141675 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141652    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:43:33.141675 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141653    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e4a2e6f-b328-492a-af9a-3fa580bd54d7" (UID: "2e4a2e6f-b328-492a-af9a-3fa580bd54d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:43:33.141850 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.141671    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:43:33.143377 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.143355    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e4a2e6f-b328-492a-af9a-3fa580bd54d7" (UID: "2e4a2e6f-b328-492a-af9a-3fa580bd54d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:43:33.143487 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.143474    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-dshm" (OuterVolumeSpecName: "dshm") pod "2e4a2e6f-b328-492a-af9a-3fa580bd54d7" (UID: "2e4a2e6f-b328-492a-af9a-3fa580bd54d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:43:33.143553 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.143471    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-kube-api-access-7zgs7" (OuterVolumeSpecName: "kube-api-access-7zgs7") pod "2e4a2e6f-b328-492a-af9a-3fa580bd54d7" (UID: "2e4a2e6f-b328-492a-af9a-3fa580bd54d7"). InnerVolumeSpecName "kube-api-access-7zgs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:43:33.242275 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.242245    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zgs7\" (UniqueName: \"kubernetes.io/projected/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-kube-api-access-7zgs7\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:43:33.242275 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.242271    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:43:33.242275 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.242281    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:43:33.242487 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.242289    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4a2e6f-b328-492a-af9a-3fa580bd54d7-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:43:33.934281 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.934199    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"
Jul 07 13:43:33.934281 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.934222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh" event={"ID":"2e4a2e6f-b328-492a-af9a-3fa580bd54d7","Type":"ContainerDied","Data":"dc6bedc3068fc06dfb80d34d17c83c1480f5ad0e6aa21e566f570c3a40361906"}
Jul 07 13:43:33.934281 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.934274    2577 scope.go:117] "RemoveContainer" containerID="1b35bd5ff246092fcf5667429adbcf295a7c25d74aac19b6d565a67b425f5c27"
Jul 07 13:43:33.951513 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.951479    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"]
Jul 07 13:43:33.952939 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:33.952919    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8880b7f7/e2e-lifecycle-model-8880b7f7-kserve-864b748667-542hh"]
Jul 07 13:43:35.514351 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:35.514318    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e4a2e6f-b328-492a-af9a-3fa580bd54d7" path="/var/lib/kubelet/pods/2e4a2e6f-b328-492a-af9a-3fa580bd54d7/volumes"
Jul 07 13:43:35.524433 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:35.524406    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:43:35.526510 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:35.526485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:43:58.499719 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:43:58.499683    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:44:14.231546 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:44:14.231508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:44:26.407486 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:44:26.407454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:45:23.621790 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:23.621760    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:45:34.062737 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:34.062700    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:45:44.501200 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.501165    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"]
Jul 07 13:45:44.501554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.501487    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e4a2e6f-b328-492a-af9a-3fa580bd54d7" containerName="main"
Jul 07 13:45:44.501554 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.501501    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e4a2e6f-b328-492a-af9a-3fa580bd54d7" containerName="main"
Jul 07 13:45:44.501626 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.501564    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e4a2e6f-b328-492a-af9a-3fa580bd54d7" containerName="main"
Jul 07 13:45:44.504461 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.504442    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.529912 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.529884    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-055c02\"/\"test-model-055c02-kserve-self-signed-certs\""
Jul 07 13:45:44.530013 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.529884    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-055c02\"/\"openshift-service-ca.crt\""
Jul 07 13:45:44.530078 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.529891    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-055c02\"/\"default-dockercfg-7dvsk\""
Jul 07 13:45:44.530133 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.529890    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-055c02\"/\"kube-root-ca.crt\""
Jul 07 13:45:44.550638 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.550619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"]
Jul 07 13:45:44.568171 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.568149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tls-certs\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.568264 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.568181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdczf\" (UniqueName: \"kubernetes.io/projected/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-kube-api-access-tdczf\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.568264 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.568203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-model-cache\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.568343 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.568273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tmp-dir\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.568343 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.568328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-dshm\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.568418 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.568350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-home\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669393 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-dshm\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669482 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-home\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669482 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tls-certs\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669482 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdczf\" (UniqueName: \"kubernetes.io/projected/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-kube-api-access-tdczf\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669482 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-model-cache\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669670 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tmp-dir\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669890 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669866    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-home\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.669988 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669926    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tmp-dir\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.670046 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.669979    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-model-cache\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.671636 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.671611    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-dshm\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.672197 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.672176    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tls-certs\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.688429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.688408    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdczf\" (UniqueName: \"kubernetes.io/projected/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-kube-api-access-tdczf\") pod \"test-model-055c02-kserve-779554d579-rmvhh\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") " pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.812862 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.812790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:44.973309 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:44.973273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"]
Jul 07 13:45:44.976051 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:45:44.976025    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf3d0f38_d0c5_4cca_9ff4_c1d0af2d3fef.slice/crio-bb16d047ee29fb73a0c44df6be6ca73d8bad7af7dc3b7372eee181c538c16493 WatchSource:0}: Error finding container bb16d047ee29fb73a0c44df6be6ca73d8bad7af7dc3b7372eee181c538c16493: Status 404 returned error can't find the container with id bb16d047ee29fb73a0c44df6be6ca73d8bad7af7dc3b7372eee181c538c16493
Jul 07 13:45:45.492632 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:45.492599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh" event={"ID":"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef","Type":"ContainerStarted","Data":"fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc"}
Jul 07 13:45:45.492632 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:45.492637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh" event={"ID":"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef","Type":"ContainerStarted","Data":"bb16d047ee29fb73a0c44df6be6ca73d8bad7af7dc3b7372eee181c538c16493"}
Jul 07 13:45:45.521326 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:45.521275    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh" podStartSLOduration=1.521256586 podStartE2EDuration="1.521256586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:45.518668866 +0000 UTC m=+1930.535825049" watchObservedRunningTime="2026-07-07 13:45:45.521256586 +0000 UTC m=+1930.538412770"
Jul 07 13:45:49.019785 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:49.019746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:45:54.813306 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:54.813275    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:54.813753 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:54.813341    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:54.825514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:54.825490    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:45:55.544115 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:45:55.544081    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:46:12.699538 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.699494    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"]
Jul 07 13:46:12.702600 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.702584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.708216 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.708195    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9285f1\"/\"default-dockercfg-ghfp9\""
Jul 07 13:46:12.708338 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.708232    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9285f1\"/\"openshift-service-ca.crt\""
Jul 07 13:46:12.709695 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.709671    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9285f1\"/\"kube-root-ca.crt\""
Jul 07 13:46:12.709858 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.709839    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9285f1\"/\"test-model-9285f1-kserve-self-signed-certs\""
Jul 07 13:46:12.729212 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.729192    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"]
Jul 07 13:46:12.742993 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.742972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-dshm\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.743074 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.743002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg4zd\" (UniqueName: \"kubernetes.io/projected/28484bfb-3adb-4675-b823-ca24ba871a0b-kube-api-access-wg4zd\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.743074 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.743023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-model-cache\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.743074 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.743059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-home\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.743183 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.743102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-tmp-dir\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.743183 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.743125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28484bfb-3adb-4675-b823-ca24ba871a0b-tls-certs\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844273 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg4zd\" (UniqueName: \"kubernetes.io/projected/28484bfb-3adb-4675-b823-ca24ba871a0b-kube-api-access-wg4zd\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844381 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-model-cache\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844381 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-home\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844381 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-tmp-dir\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844381 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28484bfb-3adb-4675-b823-ca24ba871a0b-tls-certs\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844589 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-dshm\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844761 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844733    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-model-cache\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844761 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844753    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-home\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.844923 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.844801    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-tmp-dir\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.846694 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.846673    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-dshm\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.846877 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.846860    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28484bfb-3adb-4675-b823-ca24ba871a0b-tls-certs\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:12.859183 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:12.859164    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg4zd\" (UniqueName: \"kubernetes.io/projected/28484bfb-3adb-4675-b823-ca24ba871a0b-kube-api-access-wg4zd\") pod \"test-model-9285f1-kserve-66678fb59-jgdjq\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") " pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:13.011383 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:13.011357    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:13.155906 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:13.155883    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"]
Jul 07 13:46:13.159204 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:46:13.159179    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28484bfb_3adb_4675_b823_ca24ba871a0b.slice/crio-a2de7a593bb793cd8fc6deb04ef6d47b494de4ed7c0d4e72673a2c5db0f6d1cc WatchSource:0}: Error finding container a2de7a593bb793cd8fc6deb04ef6d47b494de4ed7c0d4e72673a2c5db0f6d1cc: Status 404 returned error can't find the container with id a2de7a593bb793cd8fc6deb04ef6d47b494de4ed7c0d4e72673a2c5db0f6d1cc
Jul 07 13:46:13.602862 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:13.602828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq" event={"ID":"28484bfb-3adb-4675-b823-ca24ba871a0b","Type":"ContainerStarted","Data":"d78008df173239d636dca60166069f9944d7efe8283d1754ff7c8b6f4eacfb4c"}
Jul 07 13:46:13.603013 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:13.602870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq" event={"ID":"28484bfb-3adb-4675-b823-ca24ba871a0b","Type":"ContainerStarted","Data":"a2de7a593bb793cd8fc6deb04ef6d47b494de4ed7c0d4e72673a2c5db0f6d1cc"}
Jul 07 13:46:16.804199 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:16.804149    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq" podStartSLOduration=4.8041330129999995 podStartE2EDuration="4.804133013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:46:13.625078906 +0000 UTC m=+1958.642235089" watchObservedRunningTime="2026-07-07 13:46:16.804133013 +0000 UTC m=+1961.821289195"
Jul 07 13:46:16.804849 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:16.804830    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:46:23.011537 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:23.011494    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:23.011939 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:23.011554    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:23.023602 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:23.023581    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:23.657141 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:23.657114    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:30.185668 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:30.185636    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:46:43.730975 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:43.730936    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"]
Jul 07 13:46:44.650947 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:44.650919    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"]
Jul 07 13:46:48.236974 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:48.236943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:46:58.749974 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:58.749937    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh" podUID="bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" containerName="main" containerID="cri-o://fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc" gracePeriod=45
Jul 07 13:46:58.999909 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:58.999885    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:46:59.084697 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084628    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-dshm\") pod \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") "
Jul 07 13:46:59.084697 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084675    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-home\") pod \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") "
Jul 07 13:46:59.084697 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084693    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-model-cache\") pod \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") "
Jul 07 13:46:59.084986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084713    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tmp-dir\") pod \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") "
Jul 07 13:46:59.084986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084765    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tls-certs\") pod \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") "
Jul 07 13:46:59.084986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084784    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdczf\" (UniqueName: \"kubernetes.io/projected/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-kube-api-access-tdczf\") pod \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\" (UID: \"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef\") "
Jul 07 13:46:59.084986 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084942    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" (UID: "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.085146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.084955    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-home" (OuterVolumeSpecName: "home") pod "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" (UID: "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.085146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.085020    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" (UID: "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.085146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.085087    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:46:59.085146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.085100    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:46:59.085146 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.085110    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:46:59.086980 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.086953    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-kube-api-access-tdczf" (OuterVolumeSpecName: "kube-api-access-tdczf") pod "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" (UID: "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef"). InnerVolumeSpecName "kube-api-access-tdczf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:46:59.087061 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.087000    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-dshm" (OuterVolumeSpecName: "dshm") pod "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" (UID: "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.087061 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.087015    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" (UID: "bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:46:59.186054 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.186028    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:46:59.186054 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.186053    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdczf\" (UniqueName: \"kubernetes.io/projected/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-kube-api-access-tdczf\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:46:59.186185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.186063    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:46:59.670641 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.670581    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq" podUID="28484bfb-3adb-4675-b823-ca24ba871a0b" containerName="main" containerID="cri-o://d78008df173239d636dca60166069f9944d7efe8283d1754ff7c8b6f4eacfb4c" gracePeriod=45
Jul 07 13:46:59.799524 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.799490    2577 generic.go:356] "Generic (PLEG): container finished" podID="bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" containerID="fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc" exitCode=0
Jul 07 13:46:59.799907 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.799540    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"
Jul 07 13:46:59.799907 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.799572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh" event={"ID":"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef","Type":"ContainerDied","Data":"fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc"}
Jul 07 13:46:59.799907 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.799629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh" event={"ID":"bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef","Type":"ContainerDied","Data":"bb16d047ee29fb73a0c44df6be6ca73d8bad7af7dc3b7372eee181c538c16493"}
Jul 07 13:46:59.799907 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.799654    2577 scope.go:117] "RemoveContainer" containerID="fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc"
Jul 07 13:46:59.801488 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.801461    2577 generic.go:356] "Generic (PLEG): container finished" podID="28484bfb-3adb-4675-b823-ca24ba871a0b" containerID="d78008df173239d636dca60166069f9944d7efe8283d1754ff7c8b6f4eacfb4c" exitCode=0
Jul 07 13:46:59.801639 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.801537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq" event={"ID":"28484bfb-3adb-4675-b823-ca24ba871a0b","Type":"ContainerDied","Data":"d78008df173239d636dca60166069f9944d7efe8283d1754ff7c8b6f4eacfb4c"}
Jul 07 13:46:59.807616 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.807600    2577 scope.go:117] "RemoveContainer" containerID="fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc"
Jul 07 13:46:59.807955 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:46:59.807937    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc\": container with ID starting with fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc not found: ID does not exist" containerID="fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc"
Jul 07 13:46:59.808020 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.807962    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc"} err="failed to get container status \"fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc\": rpc error: code = NotFound desc = could not find container \"fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc\": container with ID starting with fc0aee6f9655c857027c5a586b8d95d6f65041f43e687c6b47c3f22d53e350fc not found: ID does not exist"
Jul 07 13:46:59.819339 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.819318    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"]
Jul 07 13:46:59.823429 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.823410    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-055c02/test-model-055c02-kserve-779554d579-rmvhh"]
Jul 07 13:46:59.911635 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.911614    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:46:59.990798 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.990777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg4zd\" (UniqueName: \"kubernetes.io/projected/28484bfb-3adb-4675-b823-ca24ba871a0b-kube-api-access-wg4zd\") pod \"28484bfb-3adb-4675-b823-ca24ba871a0b\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") "
Jul 07 13:46:59.990917 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.990820    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28484bfb-3adb-4675-b823-ca24ba871a0b-tls-certs\") pod \"28484bfb-3adb-4675-b823-ca24ba871a0b\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") "
Jul 07 13:46:59.990917 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.990847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-tmp-dir\") pod \"28484bfb-3adb-4675-b823-ca24ba871a0b\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") "
Jul 07 13:46:59.990917 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.990909    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-dshm\") pod \"28484bfb-3adb-4675-b823-ca24ba871a0b\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") "
Jul 07 13:46:59.991046 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.990941    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-home\") pod \"28484bfb-3adb-4675-b823-ca24ba871a0b\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") "
Jul 07 13:46:59.991046 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.990990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-model-cache\") pod \"28484bfb-3adb-4675-b823-ca24ba871a0b\" (UID: \"28484bfb-3adb-4675-b823-ca24ba871a0b\") "
Jul 07 13:46:59.991170 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.991143    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28484bfb-3adb-4675-b823-ca24ba871a0b" (UID: "28484bfb-3adb-4675-b823-ca24ba871a0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.991304 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.991268    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-home" (OuterVolumeSpecName: "home") pod "28484bfb-3adb-4675-b823-ca24ba871a0b" (UID: "28484bfb-3adb-4675-b823-ca24ba871a0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.991379 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.991328    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "28484bfb-3adb-4675-b823-ca24ba871a0b" (UID: "28484bfb-3adb-4675-b823-ca24ba871a0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.993040 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.993022    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-dshm" (OuterVolumeSpecName: "dshm") pod "28484bfb-3adb-4675-b823-ca24ba871a0b" (UID: "28484bfb-3adb-4675-b823-ca24ba871a0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:46:59.993115 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.993051    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28484bfb-3adb-4675-b823-ca24ba871a0b-kube-api-access-wg4zd" (OuterVolumeSpecName: "kube-api-access-wg4zd") pod "28484bfb-3adb-4675-b823-ca24ba871a0b" (UID: "28484bfb-3adb-4675-b823-ca24ba871a0b"). InnerVolumeSpecName "kube-api-access-wg4zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:46:59.993154 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:46:59.993125    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28484bfb-3adb-4675-b823-ca24ba871a0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28484bfb-3adb-4675-b823-ca24ba871a0b" (UID: "28484bfb-3adb-4675-b823-ca24ba871a0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:47:00.092568 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.092501    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:47:00.092568 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.092532    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg4zd\" (UniqueName: \"kubernetes.io/projected/28484bfb-3adb-4675-b823-ca24ba871a0b-kube-api-access-wg4zd\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:47:00.092568 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.092547    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28484bfb-3adb-4675-b823-ca24ba871a0b-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:47:00.092568 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.092560    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:47:00.092568 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.092571    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:47:00.092781 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.092581    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28484bfb-3adb-4675-b823-ca24ba871a0b-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:47:00.807770 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.807741    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"
Jul 07 13:47:00.807770 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.807756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq" event={"ID":"28484bfb-3adb-4675-b823-ca24ba871a0b","Type":"ContainerDied","Data":"a2de7a593bb793cd8fc6deb04ef6d47b494de4ed7c0d4e72673a2c5db0f6d1cc"}
Jul 07 13:47:00.808291 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.807798    2577 scope.go:117] "RemoveContainer" containerID="d78008df173239d636dca60166069f9944d7efe8283d1754ff7c8b6f4eacfb4c"
Jul 07 13:47:00.832880 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.832855    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"]
Jul 07 13:47:00.837666 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:00.837648    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9285f1/test-model-9285f1-kserve-66678fb59-jgdjq"]
Jul 07 13:47:01.513228 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:01.513202    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28484bfb-3adb-4675-b823-ca24ba871a0b" path="/var/lib/kubelet/pods/28484bfb-3adb-4675-b823-ca24ba871a0b/volumes"
Jul 07 13:47:01.513578 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:01.513565    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" path="/var/lib/kubelet/pods/bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef/volumes"
Jul 07 13:47:32.408829 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:32.408782    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:47:45.199251 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:45.199216    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:47:55.817525 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:47:55.817489    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:48:07.206158 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:07.206126    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:48:18.509222 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:18.509185    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:48:25.526976 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:25.526946    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:48:34.541741 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:34.541706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:48:35.547419 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:35.547388    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:48:35.550654 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:35.550633    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:48:49.026174 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.026138    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"]
Jul 07 13:48:49.026606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.026450    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28484bfb-3adb-4675-b823-ca24ba871a0b" containerName="main"
Jul 07 13:48:49.026606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.026463    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28484bfb-3adb-4675-b823-ca24ba871a0b" containerName="main"
Jul 07 13:48:49.026606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.026479    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" containerName="main"
Jul 07 13:48:49.026606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.026484    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" containerName="main"
Jul 07 13:48:49.026606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.026537    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf3d0f38-d0c5-4cca-9ff4-c1d0af2d3fef" containerName="main"
Jul 07 13:48:49.026606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.026546    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28484bfb-3adb-4675-b823-ca24ba871a0b" containerName="main"
Jul 07 13:48:49.029261 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.029243    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.034406 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.034375    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-db91fe\"/\"kube-root-ca.crt\""
Jul 07 13:48:49.034595 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.034576    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-db91fe\"/\"default-dockercfg-nq5zh\""
Jul 07 13:48:49.034683 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.034633    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-db91fe\"/\"auth-test-model-db91fe-kserve-self-signed-certs\""
Jul 07 13:48:49.034934 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.034917    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-db91fe\"/\"openshift-service-ca.crt\""
Jul 07 13:48:49.044998 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.044978    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"]
Jul 07 13:48:49.176728 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.176700    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld8xc\" (UniqueName: \"kubernetes.io/projected/09291b66-8adf-4f47-9912-c61033caed6b-kube-api-access-ld8xc\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.176872 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.176742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-dshm\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.176872 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.176758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-model-cache\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.176872 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.176832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-home\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.176997 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.176912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-tmp-dir\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.176997 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.176943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09291b66-8adf-4f47-9912-c61033caed6b-tls-certs\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.277978 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.277918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09291b66-8adf-4f47-9912-c61033caed6b-tls-certs\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.277978 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.277949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld8xc\" (UniqueName: \"kubernetes.io/projected/09291b66-8adf-4f47-9912-c61033caed6b-kube-api-access-ld8xc\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.278137 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.277983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-dshm\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.278137 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.278006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-model-cache\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.278137 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.278043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-home\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.278137 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.278086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-tmp-dir\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.278416 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.278389    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-model-cache\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.278499 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.278444    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-home\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.278499 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.278444    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-tmp-dir\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.280190 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.280164    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-dshm\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.280464 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.280448    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09291b66-8adf-4f47-9912-c61033caed6b-tls-certs\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.287861 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.287841    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld8xc\" (UniqueName: \"kubernetes.io/projected/09291b66-8adf-4f47-9912-c61033caed6b-kube-api-access-ld8xc\") pod \"auth-test-model-db91fe-kserve-7977666f79-b9bfl\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") " pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.338499 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.338476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:49.475179 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:49.475155    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"]
Jul 07 13:48:49.477607 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:48:49.477578    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09291b66_8adf_4f47_9912_c61033caed6b.slice/crio-3b2b3f855d49a6bdabb7325d9763b423264cff4cad81f816866ddcc96e92d60f WatchSource:0}: Error finding container 3b2b3f855d49a6bdabb7325d9763b423264cff4cad81f816866ddcc96e92d60f: Status 404 returned error can't find the container with id 3b2b3f855d49a6bdabb7325d9763b423264cff4cad81f816866ddcc96e92d60f
Jul 07 13:48:50.261677 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:50.261635    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl" event={"ID":"09291b66-8adf-4f47-9912-c61033caed6b","Type":"ContainerStarted","Data":"2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8"}
Jul 07 13:48:50.261677 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:50.261683    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl" event={"ID":"09291b66-8adf-4f47-9912-c61033caed6b","Type":"ContainerStarted","Data":"3b2b3f855d49a6bdabb7325d9763b423264cff4cad81f816866ddcc96e92d60f"}
Jul 07 13:48:50.296696 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:50.296650    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl" podStartSLOduration=2.296635226 podStartE2EDuration="2.296635226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:48:50.294332291 +0000 UTC m=+2115.311488474" watchObservedRunningTime="2026-07-07 13:48:50.296635226 +0000 UTC m=+2115.313791408"
Jul 07 13:48:52.126982 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:52.126946    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:48:59.339102 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:59.339071    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:59.339613 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:59.339305    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:48:59.351250 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:48:59.351226    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:49:00.311249 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:00.311219    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:49:24.932223 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:24.932189    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"]
Jul 07 13:49:24.935455 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:24.935438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:24.939951 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:24.939932    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"default-dockercfg-7qhg8\""
Jul 07 13:49:24.940189 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:24.940168    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"kube-root-ca.crt\""
Jul 07 13:49:24.940306 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:24.940217    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"auth-test-model-2170c8-kserve-self-signed-certs\""
Jul 07 13:49:24.940306 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:24.940230    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"openshift-service-ca.crt\""
Jul 07 13:49:24.952114 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:24.952091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"]
Jul 07 13:49:25.129848 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.129802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-dshm\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.129967 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.129852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-model-cache\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.129967 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.129911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8ae004b-c46e-446c-a909-7d32f00a0764-tls-certs\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.129967 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.129934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drnc6\" (UniqueName: \"kubernetes.io/projected/e8ae004b-c46e-446c-a909-7d32f00a0764-kube-api-access-drnc6\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.130090 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.129984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-home\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.130090 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.130013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-tmp-dir\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230326 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-dshm\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230326 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-model-cache\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230326 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8ae004b-c46e-446c-a909-7d32f00a0764-tls-certs\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230525 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drnc6\" (UniqueName: \"kubernetes.io/projected/e8ae004b-c46e-446c-a909-7d32f00a0764-kube-api-access-drnc6\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230525 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-home\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230525 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-tmp-dir\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230721 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230702    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-model-cache\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230777 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230759    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-home\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.230844 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.230786    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-tmp-dir\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.232545 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.232524    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-dshm\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.232900 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.232885    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8ae004b-c46e-446c-a909-7d32f00a0764-tls-certs\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.240096 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.240069    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drnc6\" (UniqueName: \"kubernetes.io/projected/e8ae004b-c46e-446c-a909-7d32f00a0764-kube-api-access-drnc6\") pod \"auth-test-model-2170c8-kserve-c7bdc697f-f7jd5\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") " pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.244716 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.244680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:25.376542 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.376509    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"]
Jul 07 13:49:25.379507 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:49:25.379470    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8ae004b_c46e_446c_a909_7d32f00a0764.slice/crio-a0d6f2f6a8a6d869877430e2c06c989183101409c246f3bb6666c3c95c194c6e WatchSource:0}: Error finding container a0d6f2f6a8a6d869877430e2c06c989183101409c246f3bb6666c3c95c194c6e: Status 404 returned error can't find the container with id a0d6f2f6a8a6d869877430e2c06c989183101409c246f3bb6666c3c95c194c6e
Jul 07 13:49:25.404931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:25.404908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5" event={"ID":"e8ae004b-c46e-446c-a909-7d32f00a0764","Type":"ContainerStarted","Data":"a0d6f2f6a8a6d869877430e2c06c989183101409c246f3bb6666c3c95c194c6e"}
Jul 07 13:49:26.410340 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:26.410231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5" event={"ID":"e8ae004b-c46e-446c-a909-7d32f00a0764","Type":"ContainerStarted","Data":"ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7"}
Jul 07 13:49:26.439322 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:26.439276    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5" podStartSLOduration=2.43926242 podStartE2EDuration="2.43926242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:49:26.438380675 +0000 UTC m=+2151.455536860" watchObservedRunningTime="2026-07-07 13:49:26.43926242 +0000 UTC m=+2151.456418603"
Jul 07 13:49:28.389796 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:28.389723    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:49:35.245507 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:35.245471    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:35.245890 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:35.245529    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:35.257665 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:35.257643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:35.458331 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:35.458307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:49:44.904983 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:49:44.904951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:50:02.097364 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:50:02.097335    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:50:15.389998 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:50:15.389963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:50:31.192444 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:50:31.192409    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:50:48.693245 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:50:48.693209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:51:04.020607 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:04.020570    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:51:18.099331 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:18.099296    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:51:22.503169 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:22.503137    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"]
Jul 07 13:51:23.269889 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:23.269859    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"]
Jul 07 13:51:33.896317 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.896281    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"]
Jul 07 13:51:33.899566 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.899549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:33.902936 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.902916    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-db91fe\"/\"sub-test-model-db91fe-kserve-self-signed-certs\""
Jul 07 13:51:33.910498 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.910474    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"]
Jul 07 13:51:33.985503 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.985472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-dshm\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:33.985639 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.985504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9907266b-dac8-497f-b25b-674d8fc1516d-tls-certs\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:33.985639 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.985536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-model-cache\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:33.985639 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.985585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-tmp-dir\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:33.985759 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.985665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-home\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:33.985759 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:33.985717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dkqn\" (UniqueName: \"kubernetes.io/projected/9907266b-dac8-497f-b25b-674d8fc1516d-kube-api-access-7dkqn\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.012097 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.012069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:51:34.086883 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.086856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-home\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.086980 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.086912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dkqn\" (UniqueName: \"kubernetes.io/projected/9907266b-dac8-497f-b25b-674d8fc1516d-kube-api-access-7dkqn\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.086980 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.086971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-dshm\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.087106 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.086995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9907266b-dac8-497f-b25b-674d8fc1516d-tls-certs\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.087106 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.087039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-model-cache\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.087106 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.087062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-tmp-dir\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.087469 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.087441    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-tmp-dir\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.087572 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.087425    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-model-cache\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.087572 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.087523    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-home\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.089579 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.089547    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9907266b-dac8-497f-b25b-674d8fc1516d-tls-certs\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.089682 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.089638    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-dshm\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.099654 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.099633    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dkqn\" (UniqueName: \"kubernetes.io/projected/9907266b-dac8-497f-b25b-674d8fc1516d-kube-api-access-7dkqn\") pod \"sub-test-model-db91fe-kserve-76f8cb8497-hxnmd\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") " pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.208365 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.208307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:34.345473 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.345445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"]
Jul 07 13:51:34.346177 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:51:34.346146    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9907266b_dac8_497f_b25b_674d8fc1516d.slice/crio-983ae36bc5bc2b893888b4373eeba46fb07f50a8a825cdb5b99ca4a58ad85d61 WatchSource:0}: Error finding container 983ae36bc5bc2b893888b4373eeba46fb07f50a8a825cdb5b99ca4a58ad85d61: Status 404 returned error can't find the container with id 983ae36bc5bc2b893888b4373eeba46fb07f50a8a825cdb5b99ca4a58ad85d61
Jul 07 13:51:34.942647 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.942612    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd" event={"ID":"9907266b-dac8-497f-b25b-674d8fc1516d","Type":"ContainerStarted","Data":"e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904"}
Jul 07 13:51:34.942647 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.942653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd" event={"ID":"9907266b-dac8-497f-b25b-674d8fc1516d","Type":"ContainerStarted","Data":"983ae36bc5bc2b893888b4373eeba46fb07f50a8a825cdb5b99ca4a58ad85d61"}
Jul 07 13:51:34.979002 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:34.978952    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd" podStartSLOduration=1.978937881 podStartE2EDuration="1.978937881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:51:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:51:34.978274513 +0000 UTC m=+2279.995430697" watchObservedRunningTime="2026-07-07 13:51:34.978937881 +0000 UTC m=+2279.996094064"
Jul 07 13:51:37.522459 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.522420    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl" podUID="09291b66-8adf-4f47-9912-c61033caed6b" containerName="main" containerID="cri-o://2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8" gracePeriod=45
Jul 07 13:51:37.772215 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.772193    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:51:37.914966 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.914891    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-dshm\") pod \"09291b66-8adf-4f47-9912-c61033caed6b\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") "
Jul 07 13:51:37.914966 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.914923    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-home\") pod \"09291b66-8adf-4f47-9912-c61033caed6b\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") "
Jul 07 13:51:37.915185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.914974    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld8xc\" (UniqueName: \"kubernetes.io/projected/09291b66-8adf-4f47-9912-c61033caed6b-kube-api-access-ld8xc\") pod \"09291b66-8adf-4f47-9912-c61033caed6b\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") "
Jul 07 13:51:37.915185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.915009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09291b66-8adf-4f47-9912-c61033caed6b-tls-certs\") pod \"09291b66-8adf-4f47-9912-c61033caed6b\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") "
Jul 07 13:51:37.915185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.915028    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-tmp-dir\") pod \"09291b66-8adf-4f47-9912-c61033caed6b\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") "
Jul 07 13:51:37.915185 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.915061    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-model-cache\") pod \"09291b66-8adf-4f47-9912-c61033caed6b\" (UID: \"09291b66-8adf-4f47-9912-c61033caed6b\") "
Jul 07 13:51:37.915378 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.915295    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-home" (OuterVolumeSpecName: "home") pod "09291b66-8adf-4f47-9912-c61033caed6b" (UID: "09291b66-8adf-4f47-9912-c61033caed6b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:37.915378 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.915309    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09291b66-8adf-4f47-9912-c61033caed6b" (UID: "09291b66-8adf-4f47-9912-c61033caed6b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:37.915378 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.915317    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-model-cache" (OuterVolumeSpecName: "model-cache") pod "09291b66-8adf-4f47-9912-c61033caed6b" (UID: "09291b66-8adf-4f47-9912-c61033caed6b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:37.917215 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.917183    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-dshm" (OuterVolumeSpecName: "dshm") pod "09291b66-8adf-4f47-9912-c61033caed6b" (UID: "09291b66-8adf-4f47-9912-c61033caed6b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:37.917333 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.917214    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09291b66-8adf-4f47-9912-c61033caed6b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09291b66-8adf-4f47-9912-c61033caed6b" (UID: "09291b66-8adf-4f47-9912-c61033caed6b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:51:37.917333 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.917244    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09291b66-8adf-4f47-9912-c61033caed6b-kube-api-access-ld8xc" (OuterVolumeSpecName: "kube-api-access-ld8xc") pod "09291b66-8adf-4f47-9912-c61033caed6b" (UID: "09291b66-8adf-4f47-9912-c61033caed6b"). InnerVolumeSpecName "kube-api-access-ld8xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:51:37.957880 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.957850    2577 generic.go:356] "Generic (PLEG): container finished" podID="09291b66-8adf-4f47-9912-c61033caed6b" containerID="2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8" exitCode=0
Jul 07 13:51:37.957990 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.957887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl" event={"ID":"09291b66-8adf-4f47-9912-c61033caed6b","Type":"ContainerDied","Data":"2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8"}
Jul 07 13:51:37.957990 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.957903    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"
Jul 07 13:51:37.957990 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.957921    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl" event={"ID":"09291b66-8adf-4f47-9912-c61033caed6b","Type":"ContainerDied","Data":"3b2b3f855d49a6bdabb7325d9763b423264cff4cad81f816866ddcc96e92d60f"}
Jul 07 13:51:37.957990 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.957943    2577 scope.go:117] "RemoveContainer" containerID="2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8"
Jul 07 13:51:37.966191 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.966172    2577 scope.go:117] "RemoveContainer" containerID="2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8"
Jul 07 13:51:37.966438 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:51:37.966418    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8\": container with ID starting with 2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8 not found: ID does not exist" containerID="2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8"
Jul 07 13:51:37.966505 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.966450    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8"} err="failed to get container status \"2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8\": rpc error: code = NotFound desc = could not find container \"2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8\": container with ID starting with 2a2fd171e4dd4eaf02ec30104966d4a29f4323fd25e4ad01daa7e14253816be8 not found: ID does not exist"
Jul 07 13:51:37.987749 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.987726    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"]
Jul 07 13:51:37.992034 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:37.992015    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/auth-test-model-db91fe-kserve-7977666f79-b9bfl"]
Jul 07 13:51:38.015606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.015583    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld8xc\" (UniqueName: \"kubernetes.io/projected/09291b66-8adf-4f47-9912-c61033caed6b-kube-api-access-ld8xc\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.015606 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.015606    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09291b66-8adf-4f47-9912-c61033caed6b-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.015729 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.015620    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.015729 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.015631    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.015729 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.015643    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.015729 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.015653    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09291b66-8adf-4f47-9912-c61033caed6b-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.289118 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.289091    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5" podUID="e8ae004b-c46e-446c-a909-7d32f00a0764" containerName="main" containerID="cri-o://ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7" gracePeriod=45
Jul 07 13:51:38.526854 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.526792    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:51:38.621496 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621437    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-home\") pod \"e8ae004b-c46e-446c-a909-7d32f00a0764\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") "
Jul 07 13:51:38.621496 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621476    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8ae004b-c46e-446c-a909-7d32f00a0764-tls-certs\") pod \"e8ae004b-c46e-446c-a909-7d32f00a0764\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") "
Jul 07 13:51:38.621496 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621491    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-tmp-dir\") pod \"e8ae004b-c46e-446c-a909-7d32f00a0764\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") "
Jul 07 13:51:38.621692 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621520    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drnc6\" (UniqueName: \"kubernetes.io/projected/e8ae004b-c46e-446c-a909-7d32f00a0764-kube-api-access-drnc6\") pod \"e8ae004b-c46e-446c-a909-7d32f00a0764\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") "
Jul 07 13:51:38.621692 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621574    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-dshm\") pod \"e8ae004b-c46e-446c-a909-7d32f00a0764\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") "
Jul 07 13:51:38.621692 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621635    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-model-cache\") pod \"e8ae004b-c46e-446c-a909-7d32f00a0764\" (UID: \"e8ae004b-c46e-446c-a909-7d32f00a0764\") "
Jul 07 13:51:38.621801 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621693    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-home" (OuterVolumeSpecName: "home") pod "e8ae004b-c46e-446c-a909-7d32f00a0764" (UID: "e8ae004b-c46e-446c-a909-7d32f00a0764"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:38.621939 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621921    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.622006 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.621943    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-model-cache" (OuterVolumeSpecName: "model-cache") pod "e8ae004b-c46e-446c-a909-7d32f00a0764" (UID: "e8ae004b-c46e-446c-a909-7d32f00a0764"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:38.622059 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.622014    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e8ae004b-c46e-446c-a909-7d32f00a0764" (UID: "e8ae004b-c46e-446c-a909-7d32f00a0764"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:38.623782 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.623762    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ae004b-c46e-446c-a909-7d32f00a0764-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e8ae004b-c46e-446c-a909-7d32f00a0764" (UID: "e8ae004b-c46e-446c-a909-7d32f00a0764"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:51:38.623903 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.623830    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-dshm" (OuterVolumeSpecName: "dshm") pod "e8ae004b-c46e-446c-a909-7d32f00a0764" (UID: "e8ae004b-c46e-446c-a909-7d32f00a0764"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:51:38.623903 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.623853    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8ae004b-c46e-446c-a909-7d32f00a0764-kube-api-access-drnc6" (OuterVolumeSpecName: "kube-api-access-drnc6") pod "e8ae004b-c46e-446c-a909-7d32f00a0764" (UID: "e8ae004b-c46e-446c-a909-7d32f00a0764"). InnerVolumeSpecName "kube-api-access-drnc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:51:38.722387 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.722366    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.722387 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.722386    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.722514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.722396    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8ae004b-c46e-446c-a909-7d32f00a0764-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.722514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.722404    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ae004b-c46e-446c-a909-7d32f00a0764-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.722514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.722412    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drnc6\" (UniqueName: \"kubernetes.io/projected/e8ae004b-c46e-446c-a909-7d32f00a0764-kube-api-access-drnc6\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:51:38.964340 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.964252    2577 generic.go:356] "Generic (PLEG): container finished" podID="e8ae004b-c46e-446c-a909-7d32f00a0764" containerID="ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7" exitCode=0
Jul 07 13:51:38.964340 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.964287    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"
Jul 07 13:51:38.964555 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.964334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5" event={"ID":"e8ae004b-c46e-446c-a909-7d32f00a0764","Type":"ContainerDied","Data":"ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7"}
Jul 07 13:51:38.964555 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.964382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5" event={"ID":"e8ae004b-c46e-446c-a909-7d32f00a0764","Type":"ContainerDied","Data":"a0d6f2f6a8a6d869877430e2c06c989183101409c246f3bb6666c3c95c194c6e"}
Jul 07 13:51:38.964555 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.964405    2577 scope.go:117] "RemoveContainer" containerID="ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7"
Jul 07 13:51:38.974422 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.974398    2577 scope.go:117] "RemoveContainer" containerID="ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7"
Jul 07 13:51:38.974713 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:51:38.974694    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7\": container with ID starting with ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7 not found: ID does not exist" containerID="ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7"
Jul 07 13:51:38.974783 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.974728    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7"} err="failed to get container status \"ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7\": rpc error: code = NotFound desc = could not find container \"ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7\": container with ID starting with ad2ebf1c63583271d8addf188bf87500f25e177d6ce4f3f6191fdd4eb16279e7 not found: ID does not exist"
Jul 07 13:51:38.997608 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:38.997585    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"]
Jul 07 13:51:39.011722 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:39.011696    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/auth-test-model-2170c8-kserve-c7bdc697f-f7jd5"]
Jul 07 13:51:39.514405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:39.514366    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09291b66-8adf-4f47-9912-c61033caed6b" path="/var/lib/kubelet/pods/09291b66-8adf-4f47-9912-c61033caed6b/volumes"
Jul 07 13:51:39.514954 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:39.514935    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8ae004b-c46e-446c-a909-7d32f00a0764" path="/var/lib/kubelet/pods/e8ae004b-c46e-446c-a909-7d32f00a0764/volumes"
Jul 07 13:51:44.208692 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:44.208661    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:44.209141 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:44.208704    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:44.221149 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:44.221124    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:44.727782 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:44.727754    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:51:45.003008 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:45.002982    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:51:57.521907 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.521869    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"]
Jul 07 13:51:57.522467 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.522304    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09291b66-8adf-4f47-9912-c61033caed6b" containerName="main"
Jul 07 13:51:57.522467 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.522325    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09291b66-8adf-4f47-9912-c61033caed6b" containerName="main"
Jul 07 13:51:57.522467 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.522342    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ae004b-c46e-446c-a909-7d32f00a0764" containerName="main"
Jul 07 13:51:57.522467 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.522351    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ae004b-c46e-446c-a909-7d32f00a0764" containerName="main"
Jul 07 13:51:57.522467 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.522458    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09291b66-8adf-4f47-9912-c61033caed6b" containerName="main"
Jul 07 13:51:57.522715 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.522474    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ae004b-c46e-446c-a909-7d32f00a0764" containerName="main"
Jul 07 13:51:57.525599 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.525576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.528907 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.528890    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"openshift-service-ca.crt\""
Jul 07 13:51:57.530158 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.530140    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"default-dockercfg-7qhg8\""
Jul 07 13:51:57.530217 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.530193    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"kube-root-ca.crt\""
Jul 07 13:51:57.530259 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.530215    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"sub-test-model-2170c8-kserve-self-signed-certs\""
Jul 07 13:51:57.547835 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.547793    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"]
Jul 07 13:51:57.672001 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.671980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-model-cache\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.672098 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.672014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7rzb\" (UniqueName: \"kubernetes.io/projected/573654eb-21ed-47ac-8b28-43dc599b2ad9-kube-api-access-r7rzb\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.672098 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.672088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-tmp-dir\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.672211 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.672122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-dshm\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.672211 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.672205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573654eb-21ed-47ac-8b28-43dc599b2ad9-tls-certs\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.672317 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.672261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-home\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.772774 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.772722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-tmp-dir\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.772774 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.772753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-dshm\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.772774 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.772770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573654eb-21ed-47ac-8b28-43dc599b2ad9-tls-certs\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.773025 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.772854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-home\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.773025 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.772907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-model-cache\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.773025 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.772943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rzb\" (UniqueName: \"kubernetes.io/projected/573654eb-21ed-47ac-8b28-43dc599b2ad9-kube-api-access-r7rzb\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.773181 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.773101    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-tmp-dir\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.773237 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.773221    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-home\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.773325 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.773305    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-model-cache\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.775520 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.775498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-dshm\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.775750 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.775732    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573654eb-21ed-47ac-8b28-43dc599b2ad9-tls-certs\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.791629 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.791608    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7rzb\" (UniqueName: \"kubernetes.io/projected/573654eb-21ed-47ac-8b28-43dc599b2ad9-kube-api-access-r7rzb\") pod \"sub-test-model-2170c8-kserve-c48464cfc-dxjbr\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") " pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.834613 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.834589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:51:57.973901 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:57.973871    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"]
Jul 07 13:51:57.978104 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:51:57.978081    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod573654eb_21ed_47ac_8b28_43dc599b2ad9.slice/crio-86ab247ea7bb7257f93a6cbeecd56263970c95a3636780fdab1c1e70e4d85395 WatchSource:0}: Error finding container 86ab247ea7bb7257f93a6cbeecd56263970c95a3636780fdab1c1e70e4d85395: Status 404 returned error can't find the container with id 86ab247ea7bb7257f93a6cbeecd56263970c95a3636780fdab1c1e70e4d85395
Jul 07 13:51:58.049169 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:58.049139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr" event={"ID":"573654eb-21ed-47ac-8b28-43dc599b2ad9","Type":"ContainerStarted","Data":"e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a"}
Jul 07 13:51:58.049266 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:58.049174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr" event={"ID":"573654eb-21ed-47ac-8b28-43dc599b2ad9","Type":"ContainerStarted","Data":"86ab247ea7bb7257f93a6cbeecd56263970c95a3636780fdab1c1e70e4d85395"}
Jul 07 13:51:58.075700 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:51:58.075659    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr" podStartSLOduration=1.075644882 podStartE2EDuration="1.075644882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:51:58.073856725 +0000 UTC m=+2303.091012907" watchObservedRunningTime="2026-07-07 13:51:58.075644882 +0000 UTC m=+2303.092801064"
Jul 07 13:52:01.696920 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:01.696890    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:52:07.834899 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:07.834859    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:52:07.835280 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:07.834911    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:52:07.847552 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:07.847530    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:52:08.111199 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:08.111116    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:52:25.611516 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:25.611473    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:52:42.480948 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:42.480910    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"]
Jul 07 13:52:43.421625 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:43.421589    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"]
Jul 07 13:52:46.713498 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:46.713465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:52:50.135987 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.135948    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"]
Jul 07 13:52:50.139374 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.139350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.146674 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.146657    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-db91fe\"/\"rate-test-model-db91fe-kserve-self-signed-certs\""
Jul 07 13:52:50.159791 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.159770    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"]
Jul 07 13:52:50.272860 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.272832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tmp-dir\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.272980 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.272877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwcm2\" (UniqueName: \"kubernetes.io/projected/ab18f96c-a6c7-46b0-8e78-2b340df03a74-kube-api-access-mwcm2\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.272980 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.272924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-home\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.273067 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.272993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-model-cache\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.273067 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.273027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tls-certs\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.273168 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.273103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-dshm\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374005 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.373977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-model-cache\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374005 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tls-certs\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374165 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-dshm\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374165 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tmp-dir\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374165 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwcm2\" (UniqueName: \"kubernetes.io/projected/ab18f96c-a6c7-46b0-8e78-2b340df03a74-kube-api-access-mwcm2\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374165 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-home\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374440 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374416    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-model-cache\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374462    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-home\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.374514 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.374494    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tmp-dir\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.376363 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.376345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-dshm\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.376579 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.376559    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tls-certs\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.391049 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.390998    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwcm2\" (UniqueName: \"kubernetes.io/projected/ab18f96c-a6c7-46b0-8e78-2b340df03a74-kube-api-access-mwcm2\") pod \"rate-test-model-db91fe-kserve-69b694bf9f-rb5fv\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") " pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.447968 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.447948    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:52:50.589526 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:50.589501    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"]
Jul 07 13:52:50.591792 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:52:50.591765    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab18f96c_a6c7_46b0_8e78_2b340df03a74.slice/crio-7f1fc5ed3a2e78d528d196ff528c17ad2a68a2d7693d043ad04ddc5b4bf62551 WatchSource:0}: Error finding container 7f1fc5ed3a2e78d528d196ff528c17ad2a68a2d7693d043ad04ddc5b4bf62551: Status 404 returned error can't find the container with id 7f1fc5ed3a2e78d528d196ff528c17ad2a68a2d7693d043ad04ddc5b4bf62551
Jul 07 13:52:51.290054 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:51.290018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv" event={"ID":"ab18f96c-a6c7-46b0-8e78-2b340df03a74","Type":"ContainerStarted","Data":"7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70"}
Jul 07 13:52:51.290398 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:51.290059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv" event={"ID":"ab18f96c-a6c7-46b0-8e78-2b340df03a74","Type":"ContainerStarted","Data":"7f1fc5ed3a2e78d528d196ff528c17ad2a68a2d7693d043ad04ddc5b4bf62551"}
Jul 07 13:52:57.500041 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.499999    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd" podUID="9907266b-dac8-497f-b25b-674d8fc1516d" containerName="main" containerID="cri-o://e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904" gracePeriod=45
Jul 07 13:52:57.749247 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.749224    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:52:57.770779 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.770700    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv" podStartSLOduration=7.770684506 podStartE2EDuration="7.770684506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:52:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:52:51.324936343 +0000 UTC m=+2356.342092525" watchObservedRunningTime="2026-07-07 13:52:57.770684506 +0000 UTC m=+2362.787840683"
Jul 07 13:52:57.830703 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.830674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dkqn\" (UniqueName: \"kubernetes.io/projected/9907266b-dac8-497f-b25b-674d8fc1516d-kube-api-access-7dkqn\") pod \"9907266b-dac8-497f-b25b-674d8fc1516d\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") "
Jul 07 13:52:57.830845 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.830729    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-model-cache\") pod \"9907266b-dac8-497f-b25b-674d8fc1516d\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") "
Jul 07 13:52:57.830845 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.830757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9907266b-dac8-497f-b25b-674d8fc1516d-tls-certs\") pod \"9907266b-dac8-497f-b25b-674d8fc1516d\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") "
Jul 07 13:52:57.830845 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.830795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-tmp-dir\") pod \"9907266b-dac8-497f-b25b-674d8fc1516d\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") "
Jul 07 13:52:57.831009 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.830848    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-dshm\") pod \"9907266b-dac8-497f-b25b-674d8fc1516d\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") "
Jul 07 13:52:57.831009 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.830909    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-home\") pod \"9907266b-dac8-497f-b25b-674d8fc1516d\" (UID: \"9907266b-dac8-497f-b25b-674d8fc1516d\") "
Jul 07 13:52:57.831009 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.830955    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9907266b-dac8-497f-b25b-674d8fc1516d" (UID: "9907266b-dac8-497f-b25b-674d8fc1516d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:57.831134 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.831069    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9907266b-dac8-497f-b25b-674d8fc1516d" (UID: "9907266b-dac8-497f-b25b-674d8fc1516d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:57.831223 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.831195    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-home" (OuterVolumeSpecName: "home") pod "9907266b-dac8-497f-b25b-674d8fc1516d" (UID: "9907266b-dac8-497f-b25b-674d8fc1516d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:57.831223 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.831210    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:57.831409 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.831244    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:57.832784 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.832761    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9907266b-dac8-497f-b25b-674d8fc1516d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9907266b-dac8-497f-b25b-674d8fc1516d" (UID: "9907266b-dac8-497f-b25b-674d8fc1516d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:52:57.832880 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.832823    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-dshm" (OuterVolumeSpecName: "dshm") pod "9907266b-dac8-497f-b25b-674d8fc1516d" (UID: "9907266b-dac8-497f-b25b-674d8fc1516d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:57.832932 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.832895    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9907266b-dac8-497f-b25b-674d8fc1516d-kube-api-access-7dkqn" (OuterVolumeSpecName: "kube-api-access-7dkqn") pod "9907266b-dac8-497f-b25b-674d8fc1516d" (UID: "9907266b-dac8-497f-b25b-674d8fc1516d"). InnerVolumeSpecName "kube-api-access-7dkqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:52:57.932633 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.932606    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:57.932633 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.932630    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9907266b-dac8-497f-b25b-674d8fc1516d-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:57.932780 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.932639    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dkqn\" (UniqueName: \"kubernetes.io/projected/9907266b-dac8-497f-b25b-674d8fc1516d-kube-api-access-7dkqn\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:57.932780 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:57.932650    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9907266b-dac8-497f-b25b-674d8fc1516d-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:58.321507 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.321474    2577 generic.go:356] "Generic (PLEG): container finished" podID="9907266b-dac8-497f-b25b-674d8fc1516d" containerID="e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904" exitCode=0
Jul 07 13:52:58.321641 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.321513    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"
Jul 07 13:52:58.321641 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.321555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd" event={"ID":"9907266b-dac8-497f-b25b-674d8fc1516d","Type":"ContainerDied","Data":"e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904"}
Jul 07 13:52:58.321641 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.321596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd" event={"ID":"9907266b-dac8-497f-b25b-674d8fc1516d","Type":"ContainerDied","Data":"983ae36bc5bc2b893888b4373eeba46fb07f50a8a825cdb5b99ca4a58ad85d61"}
Jul 07 13:52:58.321641 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.321616    2577 scope.go:117] "RemoveContainer" containerID="e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904"
Jul 07 13:52:58.330386 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.330371    2577 scope.go:117] "RemoveContainer" containerID="e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904"
Jul 07 13:52:58.330611 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:52:58.330593    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904\": container with ID starting with e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904 not found: ID does not exist" containerID="e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904"
Jul 07 13:52:58.330655 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.330620    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904"} err="failed to get container status \"e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904\": rpc error: code = NotFound desc = could not find container \"e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904\": container with ID starting with e8e38d35483fb5f2acebcd3925a993a72ef6b530b29ea4b7c6e2a709ece7a904 not found: ID does not exist"
Jul 07 13:52:58.351847 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.351822    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"]
Jul 07 13:52:58.354367 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.354348    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/sub-test-model-db91fe-kserve-76f8cb8497-hxnmd"]
Jul 07 13:52:58.440750 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.440721    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr" podUID="573654eb-21ed-47ac-8b28-43dc599b2ad9" containerName="main" containerID="cri-o://e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a" gracePeriod=45
Jul 07 13:52:58.688096 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.688075    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:52:58.839952 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.839926    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573654eb-21ed-47ac-8b28-43dc599b2ad9-tls-certs\") pod \"573654eb-21ed-47ac-8b28-43dc599b2ad9\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") "
Jul 07 13:52:58.840087 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.839964    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-dshm\") pod \"573654eb-21ed-47ac-8b28-43dc599b2ad9\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") "
Jul 07 13:52:58.840087 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.840000    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-tmp-dir\") pod \"573654eb-21ed-47ac-8b28-43dc599b2ad9\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") "
Jul 07 13:52:58.840087 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.840025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-model-cache\") pod \"573654eb-21ed-47ac-8b28-43dc599b2ad9\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") "
Jul 07 13:52:58.840087 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.840054    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7rzb\" (UniqueName: \"kubernetes.io/projected/573654eb-21ed-47ac-8b28-43dc599b2ad9-kube-api-access-r7rzb\") pod \"573654eb-21ed-47ac-8b28-43dc599b2ad9\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") "
Jul 07 13:52:58.840300 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.840097    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-home\") pod \"573654eb-21ed-47ac-8b28-43dc599b2ad9\" (UID: \"573654eb-21ed-47ac-8b28-43dc599b2ad9\") "
Jul 07 13:52:58.840379 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.840343    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "573654eb-21ed-47ac-8b28-43dc599b2ad9" (UID: "573654eb-21ed-47ac-8b28-43dc599b2ad9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:58.840379 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.840354    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-model-cache" (OuterVolumeSpecName: "model-cache") pod "573654eb-21ed-47ac-8b28-43dc599b2ad9" (UID: "573654eb-21ed-47ac-8b28-43dc599b2ad9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:58.840567 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.840549    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-home" (OuterVolumeSpecName: "home") pod "573654eb-21ed-47ac-8b28-43dc599b2ad9" (UID: "573654eb-21ed-47ac-8b28-43dc599b2ad9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:58.842080 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.842054    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/573654eb-21ed-47ac-8b28-43dc599b2ad9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "573654eb-21ed-47ac-8b28-43dc599b2ad9" (UID: "573654eb-21ed-47ac-8b28-43dc599b2ad9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:52:58.842080 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.842059    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-dshm" (OuterVolumeSpecName: "dshm") pod "573654eb-21ed-47ac-8b28-43dc599b2ad9" (UID: "573654eb-21ed-47ac-8b28-43dc599b2ad9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:52:58.842188 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.842100    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/573654eb-21ed-47ac-8b28-43dc599b2ad9-kube-api-access-r7rzb" (OuterVolumeSpecName: "kube-api-access-r7rzb") pod "573654eb-21ed-47ac-8b28-43dc599b2ad9" (UID: "573654eb-21ed-47ac-8b28-43dc599b2ad9"). InnerVolumeSpecName "kube-api-access-r7rzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:52:58.941418 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.941368    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:58.941418 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.941391    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:58.941418 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.941400    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7rzb\" (UniqueName: \"kubernetes.io/projected/573654eb-21ed-47ac-8b28-43dc599b2ad9-kube-api-access-r7rzb\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:58.941418 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.941409    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:58.941418 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.941418    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573654eb-21ed-47ac-8b28-43dc599b2ad9-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:58.941603 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:58.941426    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573654eb-21ed-47ac-8b28-43dc599b2ad9-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:52:59.328289 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.328258    2577 generic.go:356] "Generic (PLEG): container finished" podID="573654eb-21ed-47ac-8b28-43dc599b2ad9" containerID="e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a" exitCode=0
Jul 07 13:52:59.328454 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.328299    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr" event={"ID":"573654eb-21ed-47ac-8b28-43dc599b2ad9","Type":"ContainerDied","Data":"e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a"}
Jul 07 13:52:59.328454 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.328322    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"
Jul 07 13:52:59.328454 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.328334    2577 scope.go:117] "RemoveContainer" containerID="e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a"
Jul 07 13:52:59.328454 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.328325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr" event={"ID":"573654eb-21ed-47ac-8b28-43dc599b2ad9","Type":"ContainerDied","Data":"86ab247ea7bb7257f93a6cbeecd56263970c95a3636780fdab1c1e70e4d85395"}
Jul 07 13:52:59.336873 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.336853    2577 scope.go:117] "RemoveContainer" containerID="e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a"
Jul 07 13:52:59.337104 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:52:59.337089    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a\": container with ID starting with e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a not found: ID does not exist" containerID="e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a"
Jul 07 13:52:59.337152 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.337111    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a"} err="failed to get container status \"e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a\": rpc error: code = NotFound desc = could not find container \"e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a\": container with ID starting with e763e64398872ad07160983c9a9f73541fdd04f5806c0c78ec8bf0820e40e89a not found: ID does not exist"
Jul 07 13:52:59.351981 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.351959    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"]
Jul 07 13:52:59.356801 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.356783    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/sub-test-model-2170c8-kserve-c48464cfc-dxjbr"]
Jul 07 13:52:59.513345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.513319    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="573654eb-21ed-47ac-8b28-43dc599b2ad9" path="/var/lib/kubelet/pods/573654eb-21ed-47ac-8b28-43dc599b2ad9/volumes"
Jul 07 13:52:59.513685 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:52:59.513672    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9907266b-dac8-497f-b25b-674d8fc1516d" path="/var/lib/kubelet/pods/9907266b-dac8-497f-b25b-674d8fc1516d/volumes"
Jul 07 13:53:00.448561 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:00.448522    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:53:00.448951 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:00.448600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:53:00.460901 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:00.460877    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:53:01.348337 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:01.348305    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:53:01.490869 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:01.490833    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:53:14.732345 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.732314    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"]
Jul 07 13:53:14.732934 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.732761    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9907266b-dac8-497f-b25b-674d8fc1516d" containerName="main"
Jul 07 13:53:14.732934 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.732779    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9907266b-dac8-497f-b25b-674d8fc1516d" containerName="main"
Jul 07 13:53:14.732934 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.732804    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573654eb-21ed-47ac-8b28-43dc599b2ad9" containerName="main"
Jul 07 13:53:14.732934 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.732830    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="573654eb-21ed-47ac-8b28-43dc599b2ad9" containerName="main"
Jul 07 13:53:14.732934 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.732931    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="573654eb-21ed-47ac-8b28-43dc599b2ad9" containerName="main"
Jul 07 13:53:14.733192 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.732942    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9907266b-dac8-497f-b25b-674d8fc1516d" containerName="main"
Jul 07 13:53:14.735935 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.735915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.739224 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.739203    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"kube-root-ca.crt\""
Jul 07 13:53:14.740668 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.740650    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"default-dockercfg-7qhg8\""
Jul 07 13:53:14.740761 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.740685    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"openshift-service-ca.crt\""
Jul 07 13:53:14.740761 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.740704    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2170c8\"/\"rate-test-model-2170c8-kserve-self-signed-certs\""
Jul 07 13:53:14.750885 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.750864    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"]
Jul 07 13:53:14.770590 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.770566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-dshm\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.770716 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.770598    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-model-cache\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.770716 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.770616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-home\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.770716 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.770635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tls-certs\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.770716 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.770708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tmp-dir\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.770873 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.770737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djc74\" (UniqueName: \"kubernetes.io/projected/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-kube-api-access-djc74\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.871665 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.871646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-model-cache\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.871762 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.871673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-home\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.871762 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.871691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tls-certs\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.871762 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.871726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tmp-dir\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.871967 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.871766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djc74\" (UniqueName: \"kubernetes.io/projected/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-kube-api-access-djc74\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.871967 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.871850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-dshm\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.872071 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.872035    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-model-cache\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.872124 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.872080    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tmp-dir\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.872172 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.872161    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-home\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.873937 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.873914    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-dshm\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.874232 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.874214    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tls-certs\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:14.884455 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:14.884437    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djc74\" (UniqueName: \"kubernetes.io/projected/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-kube-api-access-djc74\") pod \"rate-test-model-2170c8-kserve-66d6b5594c-ln2ch\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") " pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:15.045068 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:15.045048    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:15.172324 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:15.172301    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"]
Jul 07 13:53:15.173921 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:53:15.173896    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69bfe8c3_6a0a_42bc_a2f4_ed493ea8f331.slice/crio-dda6508e63d56544c68d7257743f8c875a17368cd1c9bf61ca6f344120507bc9 WatchSource:0}: Error finding container dda6508e63d56544c68d7257743f8c875a17368cd1c9bf61ca6f344120507bc9: Status 404 returned error can't find the container with id dda6508e63d56544c68d7257743f8c875a17368cd1c9bf61ca6f344120507bc9
Jul 07 13:53:15.401229 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:15.401153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch" event={"ID":"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331","Type":"ContainerStarted","Data":"08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f"}
Jul 07 13:53:15.401229 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:15.401191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch" event={"ID":"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331","Type":"ContainerStarted","Data":"dda6508e63d56544c68d7257743f8c875a17368cd1c9bf61ca6f344120507bc9"}
Jul 07 13:53:15.423779 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:15.423737    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch" podStartSLOduration=1.4237218870000001 podStartE2EDuration="1.423721887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:53:15.422292915 +0000 UTC m=+2380.439449098" watchObservedRunningTime="2026-07-07 13:53:15.423721887 +0000 UTC m=+2380.440878070"
Jul 07 13:53:19.077902 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:19.077862    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:53:25.046169 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:25.046139    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:25.046612 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:25.046185    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:25.058220 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:25.058195    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:25.453492 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:25.453414    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:53:35.570873 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:35.570838    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:53:35.575982 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:35.575959    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:53:40.916795 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:40.916759    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:53:55.288975 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:53:55.288944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:54:02.845233 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:02.845202    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"]
Jul 07 13:54:03.686011 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:03.685984    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"]
Jul 07 13:54:08.383056 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:08.383021    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:54:17.864370 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:17.864331    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv" podUID="ab18f96c-a6c7-46b0-8e78-2b340df03a74" containerName="main" containerID="cri-o://7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70" gracePeriod=45
Jul 07 13:54:18.106736 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.106713    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:54:18.240791 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.240717    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tmp-dir\") pod \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") "
Jul 07 13:54:18.240791 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.240759    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-home\") pod \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") "
Jul 07 13:54:18.241012 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.240804    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-model-cache\") pod \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") "
Jul 07 13:54:18.241012 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.240837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tls-certs\") pod \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") "
Jul 07 13:54:18.241012 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.240855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwcm2\" (UniqueName: \"kubernetes.io/projected/ab18f96c-a6c7-46b0-8e78-2b340df03a74-kube-api-access-mwcm2\") pod \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") "
Jul 07 13:54:18.241012 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.240881    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-dshm\") pod \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\" (UID: \"ab18f96c-a6c7-46b0-8e78-2b340df03a74\") "
Jul 07 13:54:18.241175 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.241046    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab18f96c-a6c7-46b0-8e78-2b340df03a74" (UID: "ab18f96c-a6c7-46b0-8e78-2b340df03a74"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:18.241175 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.241108    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-home" (OuterVolumeSpecName: "home") pod "ab18f96c-a6c7-46b0-8e78-2b340df03a74" (UID: "ab18f96c-a6c7-46b0-8e78-2b340df03a74"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:18.241175 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.241120    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab18f96c-a6c7-46b0-8e78-2b340df03a74" (UID: "ab18f96c-a6c7-46b0-8e78-2b340df03a74"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:18.242931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.242908    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-dshm" (OuterVolumeSpecName: "dshm") pod "ab18f96c-a6c7-46b0-8e78-2b340df03a74" (UID: "ab18f96c-a6c7-46b0-8e78-2b340df03a74"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:18.243030 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.242937    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab18f96c-a6c7-46b0-8e78-2b340df03a74" (UID: "ab18f96c-a6c7-46b0-8e78-2b340df03a74"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:54:18.243030 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.243021    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab18f96c-a6c7-46b0-8e78-2b340df03a74-kube-api-access-mwcm2" (OuterVolumeSpecName: "kube-api-access-mwcm2") pod "ab18f96c-a6c7-46b0-8e78-2b340df03a74" (UID: "ab18f96c-a6c7-46b0-8e78-2b340df03a74"). InnerVolumeSpecName "kube-api-access-mwcm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:54:18.341387 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.341361    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:18.341387 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.341384    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:18.341559 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.341393    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:18.341559 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.341401    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab18f96c-a6c7-46b0-8e78-2b340df03a74-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:18.341559 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.341409    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwcm2\" (UniqueName: \"kubernetes.io/projected/ab18f96c-a6c7-46b0-8e78-2b340df03a74-kube-api-access-mwcm2\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:18.341559 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.341417    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab18f96c-a6c7-46b0-8e78-2b340df03a74-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:18.663138 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.663110    2577 generic.go:356] "Generic (PLEG): container finished" podID="ab18f96c-a6c7-46b0-8e78-2b340df03a74" containerID="7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70" exitCode=0
Jul 07 13:54:18.663244 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.663158    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"
Jul 07 13:54:18.663244 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.663193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv" event={"ID":"ab18f96c-a6c7-46b0-8e78-2b340df03a74","Type":"ContainerDied","Data":"7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70"}
Jul 07 13:54:18.663244 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.663232    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv" event={"ID":"ab18f96c-a6c7-46b0-8e78-2b340df03a74","Type":"ContainerDied","Data":"7f1fc5ed3a2e78d528d196ff528c17ad2a68a2d7693d043ad04ddc5b4bf62551"}
Jul 07 13:54:18.663357 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.663249    2577 scope.go:117] "RemoveContainer" containerID="7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70"
Jul 07 13:54:18.672280 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.672264    2577 scope.go:117] "RemoveContainer" containerID="7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70"
Jul 07 13:54:18.672535 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:54:18.672518    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70\": container with ID starting with 7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70 not found: ID does not exist" containerID="7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70"
Jul 07 13:54:18.672582 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.672541    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70"} err="failed to get container status \"7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70\": rpc error: code = NotFound desc = could not find container \"7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70\": container with ID starting with 7a50ca8e73a61c6ec30728bac7284d9b6f96b71fe5873e048f1acb9269b1cf70 not found: ID does not exist"
Jul 07 13:54:18.686919 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.686898    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"]
Jul 07 13:54:18.690982 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.690962    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-db91fe/rate-test-model-db91fe-kserve-69b694bf9f-rb5fv"]
Jul 07 13:54:18.704648 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.704599    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch" podUID="69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" containerName="main" containerID="cri-o://08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f" gracePeriod=45
Jul 07 13:54:18.943131 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:18.943110    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:54:19.045794 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.045765    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-dshm\") pod \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") "
Jul 07 13:54:19.045922 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.045808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-model-cache\") pod \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") "
Jul 07 13:54:19.045922 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.045916    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tls-certs\") pod \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") "
Jul 07 13:54:19.046002 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.045953    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djc74\" (UniqueName: \"kubernetes.io/projected/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-kube-api-access-djc74\") pod \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") "
Jul 07 13:54:19.046002 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.045991    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tmp-dir\") pod \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") "
Jul 07 13:54:19.046167 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.046016    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-model-cache" (OuterVolumeSpecName: "model-cache") pod "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" (UID: "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:19.046167 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.046071    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-home\") pod \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\" (UID: \"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331\") "
Jul 07 13:54:19.046283 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.046232    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" (UID: "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:19.046344 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.046287    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-home" (OuterVolumeSpecName: "home") pod "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" (UID: "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:19.046405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.046367    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:19.046405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.046385    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:19.046405 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.046399    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:19.047888 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.047856    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-dshm" (OuterVolumeSpecName: "dshm") pod "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" (UID: "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:54:19.047988 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.047925    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-kube-api-access-djc74" (OuterVolumeSpecName: "kube-api-access-djc74") pod "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" (UID: "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331"). InnerVolumeSpecName "kube-api-access-djc74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:54:19.047988 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.047935    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" (UID: "69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:54:19.146881 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.146856    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:19.146881 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.146876    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djc74\" (UniqueName: \"kubernetes.io/projected/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-kube-api-access-djc74\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:19.147022 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.146888    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 07 13:54:19.518791 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.518743    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab18f96c-a6c7-46b0-8e78-2b340df03a74" path="/var/lib/kubelet/pods/ab18f96c-a6c7-46b0-8e78-2b340df03a74/volumes"
Jul 07 13:54:19.668991 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.668958    2577 generic.go:356] "Generic (PLEG): container finished" podID="69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" containerID="08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f" exitCode=0
Jul 07 13:54:19.669161 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.669021    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"
Jul 07 13:54:19.669161 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.669029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch" event={"ID":"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331","Type":"ContainerDied","Data":"08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f"}
Jul 07 13:54:19.669161 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.669063    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch" event={"ID":"69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331","Type":"ContainerDied","Data":"dda6508e63d56544c68d7257743f8c875a17368cd1c9bf61ca6f344120507bc9"}
Jul 07 13:54:19.669161 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.669082    2577 scope.go:117] "RemoveContainer" containerID="08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f"
Jul 07 13:54:19.677163 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.677149    2577 scope.go:117] "RemoveContainer" containerID="08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f"
Jul 07 13:54:19.677404 ip-10-0-128-146 kubenswrapper[2577]: E0707 13:54:19.677386    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f\": container with ID starting with 08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f not found: ID does not exist" containerID="08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f"
Jul 07 13:54:19.677440 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.677413    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f"} err="failed to get container status \"08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f\": rpc error: code = NotFound desc = could not find container \"08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f\": container with ID starting with 08e43f242c5bab3448b1b15049c9adf0345956fd829629678b31dc8fbe18bc1f not found: ID does not exist"
Jul 07 13:54:19.693774 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.693749    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"]
Jul 07 13:54:19.695149 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:19.695130    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2170c8/rate-test-model-2170c8-kserve-66d6b5594c-ln2ch"]
Jul 07 13:54:21.514553 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:21.514520    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" path="/var/lib/kubelet/pods/69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331/volumes"
Jul 07 13:54:52.087887 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:52.087855    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vq6t9"]
Jul 07 13:54:56.107233 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:54:56.107203    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c784fc7ff-486df_f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b/authorino/0.log"
Jul 07 13:55:01.620833 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:01.620787    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gnj5b_e13cbdde-db69-441b-beac-6d79b8a6edbe/server/0.log"
Jul 07 13:55:01.755766 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:01.755730    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nsp4s_310e156e-0ad3-429b-a2a1-cfdaa6bd7b10/manager/0.log"
Jul 07 13:55:02.147619 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:02.147589    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-hrkm9_a75f8992-baaf-4577-a530-73dec6dec72d/copy-manifests/0.log"
Jul 07 13:55:02.165699 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:02.165660    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-hrkm9_a75f8992-baaf-4577-a530-73dec6dec72d/manager/0.log"
Jul 07 13:55:03.652207 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:03.652151    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c784fc7ff-486df_f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b/authorino/0.log"
Jul 07 13:55:04.421219 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:04.421192    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vq6t9_f3f1f12a-5579-4eb6-be7c-7ee8140abf96/limitador/0.log"
Jul 07 13:55:05.160944 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:05.160914    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-4xbgr_511ffd2b-2823-40a2-93f0-7b7670c7e011/kube-auth-proxy/0.log"
Jul 07 13:55:06.684981 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:06.684948    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-jsqqr_d9c315bd-9f27-4cdb-b0ed-fc27af79979b/main/0.log"
Jul 07 13:55:10.969200 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.969166    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g27ln/must-gather-vmkpv"]
Jul 07 13:55:10.969551 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.969457    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab18f96c-a6c7-46b0-8e78-2b340df03a74" containerName="main"
Jul 07 13:55:10.969551 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.969467    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab18f96c-a6c7-46b0-8e78-2b340df03a74" containerName="main"
Jul 07 13:55:10.969551 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.969479    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" containerName="main"
Jul 07 13:55:10.969551 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.969486    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" containerName="main"
Jul 07 13:55:10.969551 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.969541    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab18f96c-a6c7-46b0-8e78-2b340df03a74" containerName="main"
Jul 07 13:55:10.969551 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.969551    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="69bfe8c3-6a0a-42bc-a2f4-ed493ea8f331" containerName="main"
Jul 07 13:55:10.972695 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.972677    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:10.975649 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.975629    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g27ln\"/\"default-dockercfg-vwth7\""
Jul 07 13:55:10.975767 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.975721    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27ln\"/\"kube-root-ca.crt\""
Jul 07 13:55:10.976879 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.976864    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27ln\"/\"openshift-service-ca.crt\""
Jul 07 13:55:10.987656 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:10.987621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/must-gather-vmkpv"]
Jul 07 13:55:11.013585 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.013562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftmhj\" (UniqueName: \"kubernetes.io/projected/f288529e-b06b-41f8-9a61-02bb25af87b1-kube-api-access-ftmhj\") pod \"must-gather-vmkpv\" (UID: \"f288529e-b06b-41f8-9a61-02bb25af87b1\") " pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:11.013714 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.013604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f288529e-b06b-41f8-9a61-02bb25af87b1-must-gather-output\") pod \"must-gather-vmkpv\" (UID: \"f288529e-b06b-41f8-9a61-02bb25af87b1\") " pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:11.113927 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.113903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f288529e-b06b-41f8-9a61-02bb25af87b1-must-gather-output\") pod \"must-gather-vmkpv\" (UID: \"f288529e-b06b-41f8-9a61-02bb25af87b1\") " pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:11.114025 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.113958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftmhj\" (UniqueName: \"kubernetes.io/projected/f288529e-b06b-41f8-9a61-02bb25af87b1-kube-api-access-ftmhj\") pod \"must-gather-vmkpv\" (UID: \"f288529e-b06b-41f8-9a61-02bb25af87b1\") " pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:11.114216 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.114198    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f288529e-b06b-41f8-9a61-02bb25af87b1-must-gather-output\") pod \"must-gather-vmkpv\" (UID: \"f288529e-b06b-41f8-9a61-02bb25af87b1\") " pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:11.124128 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.124102    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftmhj\" (UniqueName: \"kubernetes.io/projected/f288529e-b06b-41f8-9a61-02bb25af87b1-kube-api-access-ftmhj\") pod \"must-gather-vmkpv\" (UID: \"f288529e-b06b-41f8-9a61-02bb25af87b1\") " pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:11.281558 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.281538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/must-gather-vmkpv"
Jul 07 13:55:11.411332 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.411306    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/must-gather-vmkpv"]
Jul 07 13:55:11.413499 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:55:11.413464    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf288529e_b06b_41f8_9a61_02bb25af87b1.slice/crio-5f5ae1fe1d93f727e5260afde8e6e704f1f37b6d2cb6e7ce11670314d80c4284 WatchSource:0}: Error finding container 5f5ae1fe1d93f727e5260afde8e6e704f1f37b6d2cb6e7ce11670314d80c4284: Status 404 returned error can't find the container with id 5f5ae1fe1d93f727e5260afde8e6e704f1f37b6d2cb6e7ce11670314d80c4284
Jul 07 13:55:11.415497 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.415471    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:55:11.883050 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:11.883023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/must-gather-vmkpv" event={"ID":"f288529e-b06b-41f8-9a61-02bb25af87b1","Type":"ContainerStarted","Data":"5f5ae1fe1d93f727e5260afde8e6e704f1f37b6d2cb6e7ce11670314d80c4284"}
Jul 07 13:55:12.891985 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:12.891514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/must-gather-vmkpv" event={"ID":"f288529e-b06b-41f8-9a61-02bb25af87b1","Type":"ContainerStarted","Data":"bf00cb7bd39610ad42aa85259020b40e5e971faa76b2362a4c1f04a7e8d89de6"}
Jul 07 13:55:12.891985 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:12.891572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/must-gather-vmkpv" event={"ID":"f288529e-b06b-41f8-9a61-02bb25af87b1","Type":"ContainerStarted","Data":"dc400756af39dc21a33a94ed09fab47c408f182b47973267f043cea521d9e335"}
Jul 07 13:55:12.913949 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:12.913888    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g27ln/must-gather-vmkpv" podStartSLOduration=2.148836474 podStartE2EDuration="2.913867495s" totalImagesPullingTime="765.031021ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:55:12.911790468 +0000 UTC m=+2497.928946664" watchObservedRunningTime="2026-07-07 13:55:12.913867495 +0000 UTC m=+2497.931023679"
Jul 07 13:55:13.934553 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:13.934511    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-889k8_040975fe-ab93-4030-9fe7-1fbf8b92a86a/global-pull-secret-syncer/0.log"
Jul 07 13:55:14.174789 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:14.174751    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rbcrr_fb3ea164-333c-4158-ad32-37dccca24c44/konnectivity-agent/0.log"
Jul 07 13:55:14.279709 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:14.279678    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-146.ec2.internal_0896f86a31d2ff3cbf0414b475c0fa9a/haproxy/0.log"
Jul 07 13:55:19.335800 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:19.335761    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c784fc7ff-486df_f3e38c1c-dc58-4836-8b19-a6ed4edd2f0b/authorino/0.log"
Jul 07 13:55:19.678925 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:19.678759    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vq6t9_f3f1f12a-5579-4eb6-be7c-7ee8140abf96/limitador/0.log"
Jul 07 13:55:21.583307 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:21.583272    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-kx4zp_ce6a6cc1-2d01-4e7c-b83d-4d11f6033b63/cluster-monitoring-operator/0.log"
Jul 07 13:55:21.948295 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:21.948216    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zg7jd_d22bfda0-4e00-4dac-8da9-731834d7f5d4/node-exporter/0.log"
Jul 07 13:55:21.993166 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:21.993130    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zg7jd_d22bfda0-4e00-4dac-8da9-731834d7f5d4/kube-rbac-proxy/0.log"
Jul 07 13:55:22.030087 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:22.030057    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zg7jd_d22bfda0-4e00-4dac-8da9-731834d7f5d4/init-textfile/0.log"
Jul 07 13:55:23.599467 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.599429    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"]
Jul 07 13:55:23.607539 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.607512    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.618980 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.618952    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"]
Jul 07 13:55:23.739286 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.739244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-proc\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.739710 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.739636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jbbg\" (UniqueName: \"kubernetes.io/projected/c9f40281-34e2-465c-9338-72134a98eb51-kube-api-access-4jbbg\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.739897 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.739878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-sys\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.740127 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.740110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-lib-modules\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.740338 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.740320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-podres\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.795452 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.795427    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-xprts_a814897f-909e-4249-bf67-6ab9e4c72ece/networking-console-plugin/0.log"
Jul 07 13:55:23.841286 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-proc\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.841463 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jbbg\" (UniqueName: \"kubernetes.io/projected/c9f40281-34e2-465c-9338-72134a98eb51-kube-api-access-4jbbg\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.841463 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-sys\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.841463 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-lib-modules\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.841463 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-podres\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.841678 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-podres\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.841678 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841660    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-proc\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.842008 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.841986    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-sys\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.842121 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.842103    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9f40281-34e2-465c-9338-72134a98eb51-lib-modules\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.856642 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.856543    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jbbg\" (UniqueName: \"kubernetes.io/projected/c9f40281-34e2-465c-9338-72134a98eb51-kube-api-access-4jbbg\") pod \"perf-node-gather-daemonset-hz2b7\" (UID: \"c9f40281-34e2-465c-9338-72134a98eb51\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:23.921367 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:23.921293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:24.106208 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:24.104870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"]
Jul 07 13:55:24.111042 ip-10-0-128-146 kubenswrapper[2577]: W0707 13:55:24.111017    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc9f40281_34e2_465c_9338_72134a98eb51.slice/crio-31687529744c638559890ba642028499dce5b24e1b4f1efafbb6a7089cb4dc40 WatchSource:0}: Error finding container 31687529744c638559890ba642028499dce5b24e1b4f1efafbb6a7089cb4dc40: Status 404 returned error can't find the container with id 31687529744c638559890ba642028499dce5b24e1b4f1efafbb6a7089cb4dc40
Jul 07 13:55:24.467702 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:24.467612    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/1.log"
Jul 07 13:55:24.476058 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:24.476030    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kcxzl_5f1c6542-fa3e-4921-8ce5-56a97fd9186a/console-operator/2.log"
Jul 07 13:55:24.973463 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:24.973425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7" event={"ID":"c9f40281-34e2-465c-9338-72134a98eb51","Type":"ContainerStarted","Data":"2b8ee02ae8bc2183b2fc013299857f18ff9f4ca1f827776560a444ec1a1c5811"}
Jul 07 13:55:24.973931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:24.973481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7" event={"ID":"c9f40281-34e2-465c-9338-72134a98eb51","Type":"ContainerStarted","Data":"31687529744c638559890ba642028499dce5b24e1b4f1efafbb6a7089cb4dc40"}
Jul 07 13:55:24.973931 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:24.973550    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:24.996921 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:24.996874    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7" podStartSLOduration=1.996855357 podStartE2EDuration="1.996855357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:55:24.996114566 +0000 UTC m=+2510.013270749" watchObservedRunningTime="2026-07-07 13:55:24.996855357 +0000 UTC m=+2510.014011543"
Jul 07 13:55:25.592899 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:25.592865    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-cn75k_4a90a231-23b0-42c4-802a-12f78e2403d5/volume-data-source-validator/0.log"
Jul 07 13:55:26.742513 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:26.742478    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vx4vz_f33e8d23-8a9a-4274-8bf6-4a44cd1399c8/dns/0.log"
Jul 07 13:55:26.774357 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:26.774328    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vx4vz_f33e8d23-8a9a-4274-8bf6-4a44cd1399c8/kube-rbac-proxy/0.log"
Jul 07 13:55:26.818489 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:26.818455    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gg9gk_520d0a12-2834-46f1-b42a-6f4a23c9037f/dns-node-resolver/0.log"
Jul 07 13:55:27.458686 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:27.458654    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-65558d4569-cmj2q_0961aaf9-b89c-44dd-ad00-85dd7688e07b/registry/0.log"
Jul 07 13:55:27.558628 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:27.558596    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zm2x9_0e132ef7-c007-4cec-b2f4-83ebbc940c9e/node-ca/0.log"
Jul 07 13:55:28.864789 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:28.864707    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-4xbgr_511ffd2b-2823-40a2-93f0-7b7670c7e011/kube-auth-proxy/0.log"
Jul 07 13:55:29.830040 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:29.830009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgwz9_2fece979-1d5a-4a80-aac4-7caf620121b0/serve-healthcheck-canary/0.log"
Jul 07 13:55:30.321026 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:30.320992    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-bn89d_b60fd34d-12c2-4281-8282-85e5acfbf7d2/insights-operator/0.log"
Jul 07 13:55:30.322965 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:30.322935    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-bn89d_b60fd34d-12c2-4281-8282-85e5acfbf7d2/insights-operator/1.log"
Jul 07 13:55:30.444783 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:30.444755    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2mwf7_aa650b49-e34d-4432-b6a0-e7d466eda42d/kube-rbac-proxy/0.log"
Jul 07 13:55:30.469026 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:30.468995    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2mwf7_aa650b49-e34d-4432-b6a0-e7d466eda42d/exporter/0.log"
Jul 07 13:55:30.496738 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:30.496703    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2mwf7_aa650b49-e34d-4432-b6a0-e7d466eda42d/extractor/0.log"
Jul 07 13:55:30.991400 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:30.991371    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-hz2b7"
Jul 07 13:55:33.531445 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:33.531413    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gnj5b_e13cbdde-db69-441b-beac-6d79b8a6edbe/server/0.log"
Jul 07 13:55:33.639913 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:33.639839    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nsp4s_310e156e-0ad3-429b-a2a1-cfdaa6bd7b10/manager/0.log"
Jul 07 13:55:34.157019 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:34.156993    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-4xbgr_511ffd2b-2823-40a2-93f0-7b7670c7e011/kube-auth-proxy/0.log"
Jul 07 13:55:34.157977 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:34.157950    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-hrkm9_a75f8992-baaf-4577-a530-73dec6dec72d/manager/0.log"
Jul 07 13:55:34.180450 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:34.180426    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-hrkm9_a75f8992-baaf-4577-a530-73dec6dec72d/copy-manifests/0.log"
Jul 07 13:55:35.669685 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:35.669610    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b44df4f55-xkbtg_1c30aac7-1055-498c-8e21-93e2a4352334/manager/0.log"
Jul 07 13:55:35.928542 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:35.928467    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-4xbgr_511ffd2b-2823-40a2-93f0-7b7670c7e011/kube-auth-proxy/0.log"
Jul 07 13:55:40.737705 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:40.737669    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pbj4w_c09aa4b4-dce8-4b9d-8457-d012f8246c50/migrator/0.log"
Jul 07 13:55:40.762677 ip-10-0-128-146 kubenswrapper[2577]: I0707 13:55:40.762642    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pbj4w_c09aa4b4-dce8-4b9d-8457-d012f8246c50/graceful-termination/0.log"