Jul 15 19:54:25.273558 ip-10-0-141-81 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 19:54:25.682880 ip-10-0-141-81 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:54:25.682880 ip-10-0-141-81 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 19:54:25.682880 ip-10-0-141-81 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:54:25.682880 ip-10-0-141-81 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 19:54:25.682880 ip-10-0-141-81 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:54:25.683942 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.683854    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 19:54:25.687895 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687879    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:54:25.687895 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687895    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687899    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687902    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687905    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687909    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687913    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687916    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687919    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687922    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687925    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687927    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687930    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687932    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687935    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687938    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687940    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687943    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687945    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687948    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687951    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:54:25.687958 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687953    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687959    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687962    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687974    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687977    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687979    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687982    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687985    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687987    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.687998    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688000    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688003    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688005    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688007    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688010    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688012    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688015    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688017    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688020    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688022    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688025    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:54:25.688457 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688027    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688029    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688032    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688034    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688036    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688039    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688041    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688045    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688049    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688051    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688054    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688057    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688059    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688064    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688067    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688069    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688072    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688074    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688076    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688079    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:54:25.688991 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688081    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688084    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688087    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688089    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688092    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688094    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688096    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688099    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688101    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688104    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688106    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688109    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688111    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688113    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688116    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688121    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688125    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688128    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688131    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:54:25.689488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688135    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688138    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688141    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688144    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688147    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688149    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688153    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688155    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688158    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688160    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688163    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688165    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688168    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688170    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688173    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688175    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688178    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688181    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688883    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:54:25.689937 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688890    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688893    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688895    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688898    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688901    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688904    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688906    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688909    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688912    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688914    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688917    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688919    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688922    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688927    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688931    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688934    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688937    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688940    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688942    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688945    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:54:25.690389 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688948    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688950    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688952    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688955    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688958    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688960    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688963    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688965    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688968    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688971    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688973    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688976    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688978    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688981    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688983    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688986    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688989    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688991    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688994    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688997    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:54:25.690874 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.688999    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689002    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689004    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689007    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689010    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689014    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689016    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689018    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689021    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689023    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689026    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689028    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689030    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689033    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689035    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689038    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689040    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689042    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689045    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689048    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:54:25.691408 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689051    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689053    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689056    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689058    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689060    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689063    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689065    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689068    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689071    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689075    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689079    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689082    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689084    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689087    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689090    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689092    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689097    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689100    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689103    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689106    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:54:25.691880 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689108    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689111    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689113    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689115    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689118    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689120    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689123    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689125    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689128    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689130    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689133    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689136    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689138    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689141    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689143    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689146    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689148    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689151    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689548    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689553    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:54:25.692391 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689556    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689558    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689561    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689564    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689566    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689569    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689571    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689574    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689580    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689583    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689589    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689593    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689596    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689598    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689601    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689605    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689608    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689611    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689614    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:54:25.692869 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689616    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689619    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689621    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689624    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689626    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689629    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689631    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689634    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689636    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689639    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689642    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689644    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689646    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689649    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689651    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689654    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689656    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689658    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689661    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689663    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:54:25.693340 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689665    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689670    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689672    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689675    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689678    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689680    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689683    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689685    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689688    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689690    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689693    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689695    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689698    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689700    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689702    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689705    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689708    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689710    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689713    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:54:25.693816 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689715    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689718    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689720    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689722    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689725    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689727    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689730    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689732    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689735    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689737    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689739    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689741    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689744    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689746    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689750    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689752    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689755    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689757    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689760    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689763    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:54:25.694275 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689765    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689768    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689771    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689773    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689776    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689778    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689781    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689784    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689786    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689789    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689791    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689794    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689796    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689799    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689802    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689804    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689807    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689809    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.689812    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689880    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 19:54:25.694774 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689887    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689893    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689899    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689905    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689911    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689916    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689924    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689928    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689931    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689934    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689937    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689943    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689946    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689949    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689952    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689955    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689957    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689962    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689965    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689968    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689971    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689974    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689978    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 19:54:25.695251 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689981    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689984    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689987    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689990    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689992    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689995    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.689998    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690001    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690005    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690008    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690010    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690013    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690016    2563 flags.go:64] FLAG: --enable-server="true"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690018    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690023    2563 flags.go:64] FLAG: --event-burst="100"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690027    2563 flags.go:64] FLAG: --event-qps="50"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690029    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690032    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690035    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690039    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690043    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690047    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690050    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690053    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690056    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 19:54:25.695801 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690059    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690061    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690064    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690067    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690069    2563 flags.go:64] FLAG: --feature-gates=""
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690073    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690076    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690079    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690082    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690085    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690088    2563 flags.go:64] FLAG: --help="false"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690090    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-141-81.ec2.internal"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690094    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690097    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690100    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690103    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690106    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690109    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690112    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690114    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690117    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690120    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690124    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690126    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 15 19:54:25.696487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690129    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690132    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690135    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690140    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690143    2563 flags.go:64] FLAG: --lock-file=""
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690147    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690149    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690152    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690158    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690160    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690163    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690166    2563 flags.go:64] FLAG: --logging-format="text"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690168    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690172    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690174    2563 flags.go:64] FLAG: --manifest-url=""
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690177    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690181    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690184    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690188    2563 flags.go:64] FLAG: --max-pods="110"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690191    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690194    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690196    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690200    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690202    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690205    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 19:54:25.697463 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690208    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690214    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690217    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690220    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690223    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690228    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690233    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690236    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690239    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690242    2563 flags.go:64] FLAG: --port="10250"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690246    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690250    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01a34b74f39a53623"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690253    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690256    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690259    2563 flags.go:64] FLAG: --register-node="true"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690262    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690265    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690268    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690271    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690273    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690277    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690280    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690295    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690298    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690301    2563 flags.go:64] FLAG: --runonce="false"
Jul 15 19:54:25.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690304    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690307    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690310    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690313    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690316    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690319    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690322    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690325    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690327    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690330    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690333    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690336    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690339    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690343    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690345    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690350    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690353    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690357    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690361    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690364    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690367    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690370    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690372    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690375    2563 flags.go:64] FLAG: --v="2"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690379    2563 flags.go:64] FLAG: --version="false"
Jul 15 19:54:25.699325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690383    2563 flags.go:64] FLAG: --vmodule=""
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690386    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.690390    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690545    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690550    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690553    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690556    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690559    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690561    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690564    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690567    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690570    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690572    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690575    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690578    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690581    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690583    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690586    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690588    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:54:25.700244 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690590    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690594    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690597    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690599    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690602    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690606    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690609    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690611    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690614    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690618    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690622    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690625    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690628    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690630    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690633    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690635    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690638    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690640    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690643    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:54:25.701189 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690645    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690648    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690650    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690652    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690655    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690657    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690660    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690662    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690665    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690667    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690669    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690673    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690677    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690680    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690685    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690688    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690690    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690693    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690700    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690702    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:54:25.701733 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690705    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690708    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690710    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690713    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690716    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690718    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690721    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690723    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690726    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690728    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690731    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690734    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690736    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690738    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690741    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690743    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690745    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690748    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690751    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690753    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:54:25.702567 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690757    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690760    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690763    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690766    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690768    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690771    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690775    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690778    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690781    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690784    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690788    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690790    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690793    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690795    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690798    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690801    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690803    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690807    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690810    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690812    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:54:25.703048 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690814    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690817    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690819    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.690822    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.691402    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.698874    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.698894    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.699791    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.699810    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699897    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699906    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699911    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699915    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699920    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699925    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:54:25.703879 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699929    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699933    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699938    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699942    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699947    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699951    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699955    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699960    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699971    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699975    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699979    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699984    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699988    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699992    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.699996    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700000    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700004    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700008    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700012    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700016    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:54:25.704271 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700021    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700025    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700030    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700036    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700041    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700044    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700048    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700053    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700057    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700063    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700069    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700074    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700078    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700083    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700088    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700092    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700096    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700101    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700105    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:54:25.704932 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700109    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700113    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700118    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700122    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700126    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700130    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700134    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700138    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700142    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700146    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700152    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700157    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700162    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700166    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700171    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700176    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700180    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700185    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700189    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:54:25.705444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700194    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700197    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700201    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700205    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700209    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700214    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700218    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700222    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700226    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700231    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700235    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700239    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700243    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700248    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700251    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700255    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700259    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700265    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700270    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700274    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:54:25.705897 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700279    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700302    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700306    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700311    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700314    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700319    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700323    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700327    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700331    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700335    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700338    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700343    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700347    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700351    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700354    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:54:25.706488 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.700361    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700559    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700569    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700574    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700578    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700583    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700587    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700591    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700595    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700600    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700606    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700612    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700617    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700621    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700625    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700629    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700633    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700637    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700642    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:54:25.706911 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700646    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700649    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700653    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700657    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700661    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700665    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700669    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700673    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700677    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700681    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700686    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700690    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700694    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700701    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700706    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700711    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700716    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700720    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700725    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:54:25.707386 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700730    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700734    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700739    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700743    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700748    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700752    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700757    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700762    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700766    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700771    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700775    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700779    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700783    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700788    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700792    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700795    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700800    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700804    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700809    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700812    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:54:25.707861 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700817    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700821    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700825    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700829    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700834    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700838    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700843    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700847    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700851    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700855    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700859    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700863    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700867    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700871    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700875    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700879    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700883    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700887    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700891    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700895    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:54:25.708602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700899    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700903    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700908    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700911    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700915    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700923    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700928    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700932    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700936    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700943    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700948    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700952    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700957    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700961    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700965    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700969    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700974    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700979    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700983    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:54:25.709103 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700988    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700992    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:25.700996    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.701004    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.701157    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.704475    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.705920    2563 server.go:1019] "Starting client certificate rotation"
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.706027    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:54:25.709602 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.706063    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:54:25.710343 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.710188    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:54:25.715840 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.715819    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:54:25.725537 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.725517    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 15 19:54:25.727541 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.727526    2563 log.go:25] "Validated CRI v1 image API"
Jul 15 19:54:25.728937 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.728920    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 19:54:25.731315 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.731271    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7a122bfe-af5a-466a-a64e-3efa184ab4d8:/dev/nvme0n1p3 c2db254b-1ee2-40d4-b9e7-8b9b15f8da78:/dev/nvme0n1p4]
Jul 15 19:54:25.731379 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.731314    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 19:54:25.736245 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.736141    2563 manager.go:216] Machine: {Timestamp:2026-07-15 19:54:25.734862173 +0000 UTC m=+0.336699464 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101476 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd615ff50b916a50200df44947bd2 SystemUUID:ec2cd615-ff50-b916-a502-00df44947bd2 BootID:31aca5a0-b855-48ba-b71f-15de652a3648 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d1:c0:28:9e:5b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d1:c0:28:9e:5b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:70:88:ca:b5:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 19:54:25.736245 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.736234    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 19:54:25.736396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.736326    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 19:54:25.736819 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.736800    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:54:25.738553 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.738529    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 19:54:25.738684 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.738557    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-81.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 19:54:25.739907 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.739896    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 19:54:25.739945 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.739910    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 19:54:25.739945 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.739918    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:54:25.740589 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.740579    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:54:25.741916 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.741907    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 19:54:25.744769 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.744758    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:54:25.744878 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.744868    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 19:54:25.744924 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.744915    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 19:54:25.744959 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.744927    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 19:54:25.744959 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.744938    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 19:54:25.744959 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.744946    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 15 19:54:25.744959 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.744955    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 19:54:25.745411 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.745397    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:54:25.745458 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.745423    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:54:25.747782 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.747757    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 19:54:25.749938 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.749922    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 19:54:25.750002 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.749953    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 19:54:25.750106 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750096    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 19:54:25.750150 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750111    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 19:54:25.750150 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750117    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 19:54:25.750150 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750123    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 19:54:25.750150 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750130    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 19:54:25.750150 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750136    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 19:54:25.750694 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750685    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 19:54:25.750731 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750695    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 19:54:25.750731 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750702    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 19:54:25.750731 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750708    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 19:54:25.750731 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750716    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 19:54:25.750731 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.750724    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 19:54:25.751338 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.751328    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 19:54:25.751338 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.751338    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 19:54:25.754602 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.754589    2563 server.go:1268] "Started kubelet"
Jul 15 19:54:25.754733 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.754674    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 19:54:25.754803 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.754754    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 19:54:25.754803 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.754694    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 19:54:25.754872 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.754798    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-81.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 19:54:25.754872 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.754845    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 19:54:25.754963 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.754908    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 19:54:25.755500 ip-10-0-141-81 systemd[1]: Started Kubernetes Kubelet.
Jul 15 19:54:25.755815 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.755794    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 19:54:25.756831 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.756820    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 19:54:25.760339 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.760302    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 19:54:25.761357 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.761342    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 19:54:25.761437 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.761353    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 19:54:25.761503 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.761453    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:25.761978 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.761962    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 19:54:25.762061 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.761980    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 19:54:25.762061 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.761990    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 19:54:25.762061 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762033    2563 factory.go:55] Registering systemd factory
Jul 15 19:54:25.762061 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762047    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 15 19:54:25.762061 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762059    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 19:54:25.762246 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762066    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 19:54:25.762400 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762387    2563 factory.go:153] Registering CRI-O factory
Jul 15 19:54:25.762494 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762485    2563 factory.go:223] Registration of the crio container factory successfully
Jul 15 19:54:25.762587 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762571    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 19:54:25.762651 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762601    2563 factory.go:103] Registering Raw factory
Jul 15 19:54:25.762651 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762615    2563 manager.go:1196] Started watching for new ooms in manager
Jul 15 19:54:25.762971 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.762958    2563 manager.go:349] Starting recovery of all containers
Jul 15 19:54:25.763456 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.762454    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-81.ec2.internal.18c28eaa98fe763d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-81.ec2.internal,UID:ip-10-0-141-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-81.ec2.internal,},FirstTimestamp:2026-07-15 19:54:25.754535485 +0000 UTC m=+0.356372776,LastTimestamp:2026-07-15 19:54:25.754535485 +0000 UTC m=+0.356372776,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-81.ec2.internal,}"
Jul 15 19:54:25.767017 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.766935    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 19:54:25.767017 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.767012    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 19:54:25.773121 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.773092    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vqv2h"
Jul 15 19:54:25.775181 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.775163    2563 manager.go:354] Recovery completed
Jul 15 19:54:25.779453 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.779434    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vqv2h"
Jul 15 19:54:25.779539 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.779526    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:54:25.782076 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.782057    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:54:25.782134 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.782090    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:54:25.782134 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.782103    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:54:25.782654 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.782639    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 19:54:25.782654 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.782651    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 19:54:25.782783 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.782658    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:54:25.782783 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.782663    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:54:25.783328 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.783310    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:54:25.783405 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.783338    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:54:25.783405 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.783349    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:54:25.783728 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.783717    2563 policy_none.go:49] "None policy: Start"
Jul 15 19:54:25.783765 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.783732    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 19:54:25.783765 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.783756    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 19:54:25.784024 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.783956    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-81.ec2.internal.18c28eaa9aa2ac49  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-81.ec2.internal,UID:ip-10-0-141-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-81.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-81.ec2.internal,},FirstTimestamp:2026-07-15 19:54:25.782074441 +0000 UTC m=+0.383911735,LastTimestamp:2026-07-15 19:54:25.782074441 +0000 UTC m=+0.383911735,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-81.ec2.internal,}"
Jul 15 19:54:25.784862 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.784852    2563 policy_none.go:47] "Start"
Jul 15 19:54:25.819839 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.819826    2563 manager.go:337] "Starting Device Plugin manager"
Jul 15 19:54:25.819962 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.819852    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 19:54:25.819962 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.819861    2563 server.go:87] "Starting device plugin registration server"
Jul 15 19:54:25.820103 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.820092    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 19:54:25.820141 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.820106    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 19:54:25.820185 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.820171    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 19:54:25.820278 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.820241    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 19:54:25.820278 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.820252    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 19:54:25.820803 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.820779    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 19:54:25.820871 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.820828    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:25.920699 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.920669    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:54:25.921764 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.921635    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:54:25.921764 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.921665    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:54:25.921764 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.921678    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:54:25.921764 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.921708    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-81.ec2.internal"
Jul 15 19:54:25.928152 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.928136    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-81.ec2.internal"
Jul 15 19:54:25.928219 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.928159    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-81.ec2.internal\": node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:25.945414 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.945367    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:25.945509 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.945493    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 19:54:25.946780 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.946754    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 19:54:25.946849 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.946792    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 19:54:25.946849 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.946813    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 19:54:25.946935 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:25.946851    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 19:54:25.949212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:25.949185    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:54:26.045920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.045878    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.046976 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.046958    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"]
Jul 15 19:54:26.047056 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.047028    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:54:26.049844 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.049830    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:54:26.049909 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.049857    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:54:26.049909 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.049866    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:54:26.051108 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.051097    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:54:26.051208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.051194    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.051250 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.051219    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:54:26.052731 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.052711    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:54:26.052731 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.052721    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:54:26.052855 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.052741    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:54:26.052855 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.052742    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:54:26.052855 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.052752    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:54:26.052855 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.052758    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:54:26.053933 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.053917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.054016 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.053954    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:54:26.054608 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.054596    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:54:26.054692 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.054617    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:54:26.054692 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.054628    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:54:26.064440 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.064423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b596c108751c49733a7462cca0921090-config\") pod \"kube-apiserver-proxy-ip-10-0-141-81.ec2.internal\" (UID: \"b596c108751c49733a7462cca0921090\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.064513 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.064446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a42adba9814a4962e53f94f46a160544-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"a42adba9814a4962e53f94f46a160544\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.064513 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.064463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a42adba9814a4962e53f94f46a160544-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"a42adba9814a4962e53f94f46a160544\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.070788 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.070767    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-81.ec2.internal\" not found" node="ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.075100 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.075086    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-81.ec2.internal\" not found" node="ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.146076 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.146051    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.164957 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.164935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a42adba9814a4962e53f94f46a160544-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"a42adba9814a4962e53f94f46a160544\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.165046 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.164963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b596c108751c49733a7462cca0921090-config\") pod \"kube-apiserver-proxy-ip-10-0-141-81.ec2.internal\" (UID: \"b596c108751c49733a7462cca0921090\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.165046 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.164982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a42adba9814a4962e53f94f46a160544-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"a42adba9814a4962e53f94f46a160544\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.165046 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.165018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a42adba9814a4962e53f94f46a160544-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"a42adba9814a4962e53f94f46a160544\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.165046 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.165020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a42adba9814a4962e53f94f46a160544-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"a42adba9814a4962e53f94f46a160544\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.165046 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.165035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b596c108751c49733a7462cca0921090-config\") pod \"kube-apiserver-proxy-ip-10-0-141-81.ec2.internal\" (UID: \"b596c108751c49733a7462cca0921090\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.246373 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.246322    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.346781 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.346751    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.373273 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.373256    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.378093 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.378066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:26.447044 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.447023    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.547555 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.547498    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.648029 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.648008    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.705598 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.705579    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 19:54:26.706044 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.705691    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 19:54:26.749018 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:26.748989    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:26.783346 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.783322    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:49:27 +0000 UTC" deadline="2028-04-03 10:17:50.55686571 +0000 UTC"
Jul 15 19:54:26.783346 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:26.783345    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15062h23m23.773524404s"
Jul 15 19:54:28.194056 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:28.194009    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:28.294851 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:28.294825    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:28.373829 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:28.373788    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb596c108751c49733a7462cca0921090.slice/crio-aac144ffd8229b79060fb3eb8c568bf641bf00796f3ef1c18855989615097388 WatchSource:0}: Error finding container aac144ffd8229b79060fb3eb8c568bf641bf00796f3ef1c18855989615097388: Status 404 returned error can't find the container with id aac144ffd8229b79060fb3eb8c568bf641bf00796f3ef1c18855989615097388
Jul 15 19:54:28.376829 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:28.376807    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42adba9814a4962e53f94f46a160544.slice/crio-3094dfd81ed4db32781a66780e9e3ab97d022a0ca13e95b4e65ce07acebb910c WatchSource:0}: Error finding container 3094dfd81ed4db32781a66780e9e3ab97d022a0ca13e95b4e65ce07acebb910c: Status 404 returned error can't find the container with id 3094dfd81ed4db32781a66780e9e3ab97d022a0ca13e95b4e65ce07acebb910c
Jul 15 19:54:28.380576 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.380557    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:54:28.395230 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:28.395215    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Jul 15 19:54:28.447419 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.447376    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:54:28.506959 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.506941    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:28.520019 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.519997    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:54:28.522149 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.522136    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Jul 15 19:54:28.539538 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.539521    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:54:28.622759 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.622735    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:54:28.706655 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:28.706546    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:54:29.090440 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.090356    2563 apiserver.go:52] "Watching apiserver"
Jul 15 19:54:29.097223 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.097203    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 19:54:29.098414 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.098386    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-nkvfh","openshift-multus/network-metrics-daemon-kc8fr","openshift-network-diagnostics/network-check-target-hzrbl","openshift-network-operator/iptables-alerter-c742j","kube-system/konnectivity-agent-xkvvl","openshift-cluster-node-tuning-operator/tuned-bs75x","openshift-dns/node-resolver-ljpsf","openshift-image-registry/node-ca-dhkth","openshift-multus/multus-additional-cni-plugins-zttrk","openshift-ovn-kubernetes/ovnkube-node-6sfcd","kube-system/global-pull-secret-syncer-6qndr","kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"]
Jul 15 19:54:29.099934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.099763    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.100947 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.100925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.102436 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.102400    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 19:54:29.102706 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.102689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g642n\""
Jul 15 19:54:29.102789 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.102752    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:29.102851 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.102831    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:29.102917 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.102865    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 19:54:29.103536 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.103514    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ldplg\""
Jul 15 19:54:29.103605 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.103571    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 19:54:29.103789 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.103770    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 19:54:29.103955 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.103936    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 19:54:29.104278 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.104260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.106670 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.106648    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 19:54:29.106797 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.106779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.106858 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.106829    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8772t\""
Jul 15 19:54:29.107049 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.107031    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 19:54:29.107244 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.106653    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 19:54:29.107299 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.107243    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:54:29.107354 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.107320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.108279 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.108260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.110390 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.109584    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-89xvb\""
Jul 15 19:54:29.110390 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.109656    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 19:54:29.110390 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.109852    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:54:29.110390 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.109873    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2fkml\""
Jul 15 19:54:29.110390 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110007    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 19:54:29.110390 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110079    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 19:54:29.110390 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.110815 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110565    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9ptfk\""
Jul 15 19:54:29.110815 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110585    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 19:54:29.110815 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110710    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 19:54:29.110815 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110755    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 19:54:29.111007 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.110944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 19:54:29.111222 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.111204    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 19:54:29.113357 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.113118    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:29.113357 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.113176    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:29.113357 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.113318    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zhzz2\""
Jul 15 19:54:29.113357 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.113330    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 19:54:29.114758 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.114739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.116512 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.116495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.116608 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.116545    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.117771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd71fa81-6083-41ca-b02a-1b925656891b-tmp-dir\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-device-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-sys\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-lib-modules\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-etc-selinux\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:29.118681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-kubernetes\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118797    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-run\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-sys-fs\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a9a4ff3-7833-4ced-bbdd-504024118cb4-host-slash\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz9z2\" (UniqueName: \"kubernetes.io/projected/8a9a4ff3-7833-4ced-bbdd-504024118cb4-kube-api-access-wz9z2\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118912    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-systemd\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-tuned\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.118978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd71fa81-6083-41ca-b02a-1b925656891b-hosts-file\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-modprobe-d\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngdck\" (UniqueName: \"kubernetes.io/projected/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-kube-api-access-ngdck\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tksqx\" (UniqueName: \"kubernetes.io/projected/d7124dc7-022e-424a-8dfd-e4b1b22b5044-kube-api-access-tksqx\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.119104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/835e5e3e-cda1-4cde-8e93-153ff2732d8a-agent-certs\") pod \"konnectivity-agent-xkvvl\" (UID: \"835e5e3e-cda1-4cde-8e93-153ff2732d8a\") " pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysctl-conf\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119161    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-var-lib-kubelet\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-tmp\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-socket-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-registration-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a9a4ff3-7833-4ced-bbdd-504024118cb4-iptables-alerter-script\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/835e5e3e-cda1-4cde-8e93-153ff2732d8a-konnectivity-ca\") pod \"konnectivity-agent-xkvvl\" (UID: \"835e5e3e-cda1-4cde-8e93-153ff2732d8a\") " pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysconfig\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.119604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2ffg\" (UniqueName: \"kubernetes.io/projected/bd71fa81-6083-41ca-b02a-1b925656891b-kube-api-access-j2ffg\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.120059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysctl-d\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.120059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.119666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-host\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.122396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.122376    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 19:54:29.123199 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.123169    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 19:54:29.123585 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.123567    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 19:54:29.123832 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.123794    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 19:54:29.123916 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.123875    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xw9vs\""
Jul 15 19:54:29.124000 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.123985    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 19:54:29.124085 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.124071    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nxm5c\""
Jul 15 19:54:29.124185 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.124171    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 19:54:29.124414 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.124393    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 19:54:29.124489 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.124412    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 19:54:29.126276 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.126254    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:54:29.169200 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.169179    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xdt8h"
Jul 15 19:54:29.177816 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.177795    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xdt8h"
Jul 15 19:54:29.208209 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.208188    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 19:54:29.220431 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42abef4d-d44e-48a7-8847-904feec67e60-ovn-node-metrics-cert\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.220526 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-cni-binary-copy\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.220526 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrns7\" (UniqueName: \"kubernetes.io/projected/ebf623c0-004d-49f5-81f2-1855e3aba827-kube-api-access-wrns7\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.220526 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz9z2\" (UniqueName: \"kubernetes.io/projected/8a9a4ff3-7833-4ced-bbdd-504024118cb4-kube-api-access-wz9z2\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.220674 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-tuned\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.220674 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-cni-bin\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.220674 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-systemd-units\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.220674 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-modprobe-d\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.220674 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngdck\" (UniqueName: \"kubernetes.io/projected/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-kube-api-access-ngdck\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.220674 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-kubelet\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.220674 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95419d71-9a42-48ac-a319-70c556aa50bf-kubelet-config\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0728008-4dd7-4bb5-8220-1679c4f82a4a-host\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tksqx\" (UniqueName: \"kubernetes.io/projected/d7124dc7-022e-424a-8dfd-e4b1b22b5044-kube-api-access-tksqx\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/835e5e3e-cda1-4cde-8e93-153ff2732d8a-agent-certs\") pod \"konnectivity-agent-xkvvl\" (UID: \"835e5e3e-cda1-4cde-8e93-153ff2732d8a\") " pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-daemon-config\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220771    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-modprobe-d\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-multus-certs\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-run-ovn-kubernetes\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-var-lib-kubelet\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-tmp\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220865    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-socket-dir-parent\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-etc-kubernetes\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-var-lib-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.220999 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.220952    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-node-log\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-var-lib-kubelet\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-socket-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-registration-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a9a4ff3-7833-4ced-bbdd-504024118cb4-iptables-alerter-script\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-system-cni-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-kubelet\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-systemd\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-run-netns\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-slash\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-socket-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18c22562-fc3d-4b9c-aa81-02d56e4775c8-cni-binary-copy\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-env-overrides\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-registration-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.221575 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-system-cni-dir\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-lib-modules\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-os-release\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-cni-multus\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-lib-modules\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-ovnkube-script-lib\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-run\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221765    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-run\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-netns\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a9a4ff3-7833-4ced-bbdd-504024118cb4-iptables-alerter-script\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj6zj\" (UniqueName: \"kubernetes.io/projected/760b10f9-2f25-4197-9678-518c2f396178-kube-api-access-qj6zj\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl8j6\" (UniqueName: \"kubernetes.io/projected/42abef4d-d44e-48a7-8847-904feec67e60-kube-api-access-vl8j6\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-sys-fs\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.222212 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a9a4ff3-7833-4ced-bbdd-504024118cb4-host-slash\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.221995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-systemd\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222015    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-sys-fs\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f7kd\" (UniqueName: \"kubernetes.io/projected/18c22562-fc3d-4b9c-aa81-02d56e4775c8-kube-api-access-2f7kd\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a9a4ff3-7833-4ced-bbdd-504024118cb4-host-slash\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-ovn\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-log-socket\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-systemd\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95419d71-9a42-48ac-a319-70c556aa50bf-dbus\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd71fa81-6083-41ca-b02a-1b925656891b-hosts-file\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-etc-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-ovnkube-config\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd71fa81-6083-41ca-b02a-1b925656891b-hosts-file\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h469n\" (UniqueName: \"kubernetes.io/projected/e0728008-4dd7-4bb5-8220-1679c4f82a4a-kube-api-access-h469n\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-cnibin\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysctl-conf\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-cnibin\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-os-release\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0728008-4dd7-4bb5-8220-1679c4f82a4a-serviceca\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/835e5e3e-cda1-4cde-8e93-153ff2732d8a-konnectivity-ca\") pod \"konnectivity-agent-xkvvl\" (UID: \"835e5e3e-cda1-4cde-8e93-153ff2732d8a\") " pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysconfig\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysctl-conf\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-k8s-cni-cncf-io\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222498    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysconfig\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-cni-netd\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2ffg\" (UniqueName: \"kubernetes.io/projected/bd71fa81-6083-41ca-b02a-1b925656891b-kube-api-access-j2ffg\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysctl-d\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-host\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-host\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-sysctl-d\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-hostroot\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-conf-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.223604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd71fa81-6083-41ca-b02a-1b925656891b-tmp-dir\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-device-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.222974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-sys\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-cni-bin\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-device-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-etc-selinux\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-kubernetes\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-cni-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7124dc7-022e-424a-8dfd-e4b1b22b5044-etc-selinux\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/835e5e3e-cda1-4cde-8e93-153ff2732d8a-konnectivity-ca\") pod \"konnectivity-agent-xkvvl\" (UID: \"835e5e3e-cda1-4cde-8e93-153ff2732d8a\") " pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-sys\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-kubernetes\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.224396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.223281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd71fa81-6083-41ca-b02a-1b925656891b-tmp-dir\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.225079 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.224495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-etc-tuned\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.225079 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.224550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-tmp\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.225079 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.224707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/835e5e3e-cda1-4cde-8e93-153ff2732d8a-agent-certs\") pod \"konnectivity-agent-xkvvl\" (UID: \"835e5e3e-cda1-4cde-8e93-153ff2732d8a\") " pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.234194 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.234174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngdck\" (UniqueName: \"kubernetes.io/projected/bf9871d5-3f5b-4020-a852-1a0ff24d99a3-kube-api-access-ngdck\") pod \"tuned-bs75x\" (UID: \"bf9871d5-3f5b-4020-a852-1a0ff24d99a3\") " pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.235170 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.235140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tksqx\" (UniqueName: \"kubernetes.io/projected/d7124dc7-022e-424a-8dfd-e4b1b22b5044-kube-api-access-tksqx\") pod \"aws-ebs-csi-driver-node-zwxrn\" (UID: \"d7124dc7-022e-424a-8dfd-e4b1b22b5044\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.240109 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.240088    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:54:29.240191 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.240118    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:54:29.240191 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.240131    2563 projected.go:196] Error preparing data for projected volume kube-api-access-ldmbd for pod openshift-network-diagnostics/network-check-target-hzrbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:29.240191 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.240187    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd podName:1b9b5d7f-bd90-4818-9997-193de9a8eb7c nodeName:}" failed. No retries permitted until 2026-07-15 19:54:29.740167938 +0000 UTC m=+2.997570106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmbd" (UniqueName: "kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd") pod "network-check-target-hzrbl" (UID: "1b9b5d7f-bd90-4818-9997-193de9a8eb7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:29.242786 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.242763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2ffg\" (UniqueName: \"kubernetes.io/projected/bd71fa81-6083-41ca-b02a-1b925656891b-kube-api-access-j2ffg\") pod \"node-resolver-ljpsf\" (UID: \"bd71fa81-6083-41ca-b02a-1b925656891b\") " pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.243981 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.243944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz9z2\" (UniqueName: \"kubernetes.io/projected/8a9a4ff3-7833-4ced-bbdd-504024118cb4-kube-api-access-wz9z2\") pod \"iptables-alerter-c742j\" (UID: \"8a9a4ff3-7833-4ced-bbdd-504024118cb4\") " pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.299012 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.298955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" event={"ID":"a42adba9814a4962e53f94f46a160544","Type":"ContainerStarted","Data":"3094dfd81ed4db32781a66780e9e3ab97d022a0ca13e95b4e65ce07acebb910c"}
Jul 15 19:54:29.300242 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.300180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal" event={"ID":"b596c108751c49733a7462cca0921090","Type":"ContainerStarted","Data":"aac144ffd8229b79060fb3eb8c568bf641bf00796f3ef1c18855989615097388"}
Jul 15 19:54:29.323377 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-daemon-config\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323377 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-multus-certs\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323555 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-run-ovn-kubernetes\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.323555 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-socket-dir-parent\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323555 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-socket-dir-parent\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323555 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323485    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-multus-certs\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323555 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323485    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-run-ovn-kubernetes\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.323555 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-etc-kubernetes\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-var-lib-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-node-log\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-etc-kubernetes\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-var-lib-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-system-cni-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323747    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-node-log\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323792    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-system-cni-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.323918 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.323704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.324426 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-daemon-config\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.324749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-kubelet\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.324749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-systemd\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.324749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-run-netns\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.324749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-systemd\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.324749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-kubelet\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-slash\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-run-netns\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-slash\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18c22562-fc3d-4b9c-aa81-02d56e4775c8-cni-binary-copy\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-env-overrides\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324877    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-system-cni-dir\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-os-release\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324924    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-cni-multus\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.324991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-ovnkube-script-lib\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-netns\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj6zj\" (UniqueName: \"kubernetes.io/projected/760b10f9-2f25-4197-9678-518c2f396178-kube-api-access-qj6zj\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl8j6\" (UniqueName: \"kubernetes.io/projected/42abef4d-d44e-48a7-8847-904feec67e60-kube-api-access-vl8j6\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f7kd\" (UniqueName: \"kubernetes.io/projected/18c22562-fc3d-4b9c-aa81-02d56e4775c8-kube-api-access-2f7kd\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325208 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-ovn\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-log-socket\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18c22562-fc3d-4b9c-aa81-02d56e4775c8-cni-binary-copy\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-system-cni-dir\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-os-release\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-cni-multus\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95419d71-9a42-48ac-a319-70c556aa50bf-dbus\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95419d71-9a42-48ac-a319-70c556aa50bf-dbus\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-etc-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-ovnkube-config\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h469n\" (UniqueName: \"kubernetes.io/projected/e0728008-4dd7-4bb5-8220-1679c4f82a4a-kube-api-access-h469n\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-cnibin\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325612    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-cnibin\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-os-release\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0728008-4dd7-4bb5-8220-1679c4f82a4a-serviceca\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-k8s-cni-cncf-io\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.325997 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.325420    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-cni-netd\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-hostroot\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-ovnkube-script-lib\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.325896    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:29.825883148 +0000 UTC m=+3.083285309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-etc-openvswitch\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-cnibin\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-conf-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325958    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-conf-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.325829    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.325995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-cni-bin\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-cni-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42abef4d-d44e-48a7-8847-904feec67e60-ovn-node-metrics-cert\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.326061    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret podName:95419d71-9a42-48ac-a319-70c556aa50bf nodeName:}" failed. No retries permitted until 2026-07-15 19:54:29.826041342 +0000 UTC m=+3.083443511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret") pod "global-pull-secret-syncer-6qndr" (UID: "95419d71-9a42-48ac-a319-70c556aa50bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-cni-binary-copy\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.326934 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrns7\" (UniqueName: \"kubernetes.io/projected/ebf623c0-004d-49f5-81f2-1855e3aba827-kube-api-access-wrns7\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-cni-bin\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-systemd-units\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-kubelet\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95419d71-9a42-48ac-a319-70c556aa50bf-kubelet-config\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-ovnkube-config\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0728008-4dd7-4bb5-8220-1679c4f82a4a-host\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0728008-4dd7-4bb5-8220-1679c4f82a4a-host\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-run-ovn\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326414    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-netns\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-log-socket\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326737    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-hostroot\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e0728008-4dd7-4bb5-8220-1679c4f82a4a-serviceca\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-cni-bin\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-run-k8s-cni-cncf-io\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-multus-cni-dir\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-host-cni-netd\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.327747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-cnibin\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebf623c0-004d-49f5-81f2-1855e3aba827-os-release\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.326962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-cni-bin\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.327004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c22562-fc3d-4b9c-aa81-02d56e4775c8-host-var-lib-kubelet\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.327035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-cni-binary-copy\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.327050    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95419d71-9a42-48ac-a319-70c556aa50bf-kubelet-config\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.327046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ebf623c0-004d-49f5-81f2-1855e3aba827-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.327060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42abef4d-d44e-48a7-8847-904feec67e60-systemd-units\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.328656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.327270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42abef4d-d44e-48a7-8847-904feec67e60-env-overrides\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.329039 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.328919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42abef4d-d44e-48a7-8847-904feec67e60-ovn-node-metrics-cert\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.338747 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.338705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl8j6\" (UniqueName: \"kubernetes.io/projected/42abef4d-d44e-48a7-8847-904feec67e60-kube-api-access-vl8j6\") pod \"ovnkube-node-6sfcd\" (UID: \"42abef4d-d44e-48a7-8847-904feec67e60\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.339012 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.338988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h469n\" (UniqueName: \"kubernetes.io/projected/e0728008-4dd7-4bb5-8220-1679c4f82a4a-kube-api-access-h469n\") pod \"node-ca-dhkth\" (UID: \"e0728008-4dd7-4bb5-8220-1679c4f82a4a\") " pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.342378 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.339781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f7kd\" (UniqueName: \"kubernetes.io/projected/18c22562-fc3d-4b9c-aa81-02d56e4775c8-kube-api-access-2f7kd\") pod \"multus-nkvfh\" (UID: \"18c22562-fc3d-4b9c-aa81-02d56e4775c8\") " pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.342378 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.340428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrns7\" (UniqueName: \"kubernetes.io/projected/ebf623c0-004d-49f5-81f2-1855e3aba827-kube-api-access-wrns7\") pod \"multus-additional-cni-plugins-zttrk\" (UID: \"ebf623c0-004d-49f5-81f2-1855e3aba827\") " pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.342378 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.340459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj6zj\" (UniqueName: \"kubernetes.io/projected/760b10f9-2f25-4197-9678-518c2f396178-kube-api-access-qj6zj\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:29.386053 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.386016    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:54:29.412833 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.412802    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ljpsf"
Jul 15 19:54:29.425360 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.425336    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn"
Jul 15 19:54:29.434013 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.433991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c742j"
Jul 15 19:54:29.440808 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.440793    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:29.449286 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.449268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bs75x"
Jul 15 19:54:29.455869 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.455849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zttrk"
Jul 15 19:54:29.462821 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.462803    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nkvfh"
Jul 15 19:54:29.470626 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.470605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:29.477337 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.477318    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dhkth"
Jul 15 19:54:29.829622 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.829589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:29.829802 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.829634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:29.829802 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:29.829675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:29.829802 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829771    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:29.829953 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829803    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:29.829953 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829818    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:54:29.829953 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829840    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:54:29.829953 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829842    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:30.829821244 +0000 UTC m=+4.087223416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:29.829953 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829852    2563 projected.go:196] Error preparing data for projected volume kube-api-access-ldmbd for pod openshift-network-diagnostics/network-check-target-hzrbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:29.829953 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829863    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret podName:95419d71-9a42-48ac-a319-70c556aa50bf nodeName:}" failed. No retries permitted until 2026-07-15 19:54:30.829853189 +0000 UTC m=+4.087255349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret") pod "global-pull-secret-syncer-6qndr" (UID: "95419d71-9a42-48ac-a319-70c556aa50bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:29.829953 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:29.829895    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd podName:1b9b5d7f-bd90-4818-9997-193de9a8eb7c nodeName:}" failed. No retries permitted until 2026-07-15 19:54:30.829884125 +0000 UTC m=+4.087286281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmbd" (UniqueName: "kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd") pod "network-check-target-hzrbl" (UID: "1b9b5d7f-bd90-4818-9997-193de9a8eb7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:30.179868 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:30.179793    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:49:29 +0000 UTC" deadline="2028-02-16 22:13:21.836223424 +0000 UTC"
Jul 15 19:54:30.179868 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:30.179824    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13946h18m51.656403181s"
Jul 15 19:54:30.292470 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:30.292440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:30.292622 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.292544    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:30.353281 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.353257    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9871d5_3f5b_4020_a852_1a0ff24d99a3.slice/crio-6c859b4bc2f5c3fd581b7737bfee046286ded31102eb1cdfb8a136611ebc6c66 WatchSource:0}: Error finding container 6c859b4bc2f5c3fd581b7737bfee046286ded31102eb1cdfb8a136611ebc6c66: Status 404 returned error can't find the container with id 6c859b4bc2f5c3fd581b7737bfee046286ded31102eb1cdfb8a136611ebc6c66
Jul 15 19:54:30.355029 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.355003    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42abef4d_d44e_48a7_8847_904feec67e60.slice/crio-95d6e977449761f2eccbf49b1af7b06efe6109ffb89588a93f0af89358d9d393 WatchSource:0}: Error finding container 95d6e977449761f2eccbf49b1af7b06efe6109ffb89588a93f0af89358d9d393: Status 404 returned error can't find the container with id 95d6e977449761f2eccbf49b1af7b06efe6109ffb89588a93f0af89358d9d393
Jul 15 19:54:30.356291 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.356208    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd71fa81_6083_41ca_b02a_1b925656891b.slice/crio-874b5090c55543357bf689d6999760528364949caf8eb92df7db83c2ace2e575 WatchSource:0}: Error finding container 874b5090c55543357bf689d6999760528364949caf8eb92df7db83c2ace2e575: Status 404 returned error can't find the container with id 874b5090c55543357bf689d6999760528364949caf8eb92df7db83c2ace2e575
Jul 15 19:54:30.358250 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.357915    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0728008_4dd7_4bb5_8220_1679c4f82a4a.slice/crio-34da2426f8e21ae7476a46d12ccdcc02a36bc2877eef3e8f612c4fdaf09d38ca WatchSource:0}: Error finding container 34da2426f8e21ae7476a46d12ccdcc02a36bc2877eef3e8f612c4fdaf09d38ca: Status 404 returned error can't find the container with id 34da2426f8e21ae7476a46d12ccdcc02a36bc2877eef3e8f612c4fdaf09d38ca
Jul 15 19:54:30.360706 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.360678    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7124dc7_022e_424a_8dfd_e4b1b22b5044.slice/crio-638211cc387939c49997c6c8794e40b2e8ceee9cd6a79101fab735eb6d5500c2 WatchSource:0}: Error finding container 638211cc387939c49997c6c8794e40b2e8ceee9cd6a79101fab735eb6d5500c2: Status 404 returned error can't find the container with id 638211cc387939c49997c6c8794e40b2e8ceee9cd6a79101fab735eb6d5500c2
Jul 15 19:54:30.361592 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.361565    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9a4ff3_7833_4ced_bbdd_504024118cb4.slice/crio-d7244a2944bf4bda03a29ca8ce1112d9cc923a965ea7a75e071fa2462a141659 WatchSource:0}: Error finding container d7244a2944bf4bda03a29ca8ce1112d9cc923a965ea7a75e071fa2462a141659: Status 404 returned error can't find the container with id d7244a2944bf4bda03a29ca8ce1112d9cc923a965ea7a75e071fa2462a141659
Jul 15 19:54:30.363546 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.363509    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835e5e3e_cda1_4cde_8e93_153ff2732d8a.slice/crio-a3786f573c6db70aadca7adc7a7bbde516d7bc7d89ac1bc0c81383c6859039db WatchSource:0}: Error finding container a3786f573c6db70aadca7adc7a7bbde516d7bc7d89ac1bc0c81383c6859039db: Status 404 returned error can't find the container with id a3786f573c6db70aadca7adc7a7bbde516d7bc7d89ac1bc0c81383c6859039db
Jul 15 19:54:30.364577 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.364518    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf623c0_004d_49f5_81f2_1855e3aba827.slice/crio-961d6a2f824400ed0297f134e5644700fffcde2a29130b1c69b5abc48f2f6226 WatchSource:0}: Error finding container 961d6a2f824400ed0297f134e5644700fffcde2a29130b1c69b5abc48f2f6226: Status 404 returned error can't find the container with id 961d6a2f824400ed0297f134e5644700fffcde2a29130b1c69b5abc48f2f6226
Jul 15 19:54:30.367064 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:54:30.366786    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c22562_fc3d_4b9c_aa81_02d56e4775c8.slice/crio-9093d787f2111f278afdcfbdf4a7ac891319575b39dd14f1007bfe2beba1eac1 WatchSource:0}: Error finding container 9093d787f2111f278afdcfbdf4a7ac891319575b39dd14f1007bfe2beba1eac1: Status 404 returned error can't find the container with id 9093d787f2111f278afdcfbdf4a7ac891319575b39dd14f1007bfe2beba1eac1
Jul 15 19:54:30.849846 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:30.849519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:30.849846 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.849782    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:30.850063 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.849854    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret podName:95419d71-9a42-48ac-a319-70c556aa50bf nodeName:}" failed. No retries permitted until 2026-07-15 19:54:32.849833502 +0000 UTC m=+6.107235663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret") pod "global-pull-secret-syncer-6qndr" (UID: "95419d71-9a42-48ac-a319-70c556aa50bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:30.850063 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:30.849903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:30.850063 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:30.849937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:30.850063 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.850049    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:54:30.850264 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.850067    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:54:30.850264 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.850080    2563 projected.go:196] Error preparing data for projected volume kube-api-access-ldmbd for pod openshift-network-diagnostics/network-check-target-hzrbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:30.850264 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.850114    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd podName:1b9b5d7f-bd90-4818-9997-193de9a8eb7c nodeName:}" failed. No retries permitted until 2026-07-15 19:54:32.8501041 +0000 UTC m=+6.107506257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmbd" (UniqueName: "kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd") pod "network-check-target-hzrbl" (UID: "1b9b5d7f-bd90-4818-9997-193de9a8eb7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:30.850264 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.850169    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:30.850264 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:30.850191    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:32.850184263 +0000 UTC m=+6.107586420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:31.180951 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.180747    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:49:29 +0000 UTC" deadline="2028-01-14 10:23:13.781991133 +0000 UTC"
Jul 15 19:54:31.180951 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.180858    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13142h28m42.601138492s"
Jul 15 19:54:31.295506 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.294698    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:31.295506 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:31.294853    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:31.295506 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.294937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:31.295506 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:31.295013    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:31.319820 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.319788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal" event={"ID":"b596c108751c49733a7462cca0921090","Type":"ContainerStarted","Data":"7c04d188f4aa9129a5a001b124903d2de38f7f0e79a3c3d25b23d103595fca7f"}
Jul 15 19:54:31.328989 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.328945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c742j" event={"ID":"8a9a4ff3-7833-4ced-bbdd-504024118cb4","Type":"ContainerStarted","Data":"d7244a2944bf4bda03a29ca8ce1112d9cc923a965ea7a75e071fa2462a141659"}
Jul 15 19:54:31.334278 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.334250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nkvfh" event={"ID":"18c22562-fc3d-4b9c-aa81-02d56e4775c8","Type":"ContainerStarted","Data":"9093d787f2111f278afdcfbdf4a7ac891319575b39dd14f1007bfe2beba1eac1"}
Jul 15 19:54:31.337931 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.337879    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal" podStartSLOduration=3.337861248 podStartE2EDuration="3.337861248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:31.336686217 +0000 UTC m=+4.594088400" watchObservedRunningTime="2026-07-15 19:54:31.337861248 +0000 UTC m=+4.595263427"
Jul 15 19:54:31.338372 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.338317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkvvl" event={"ID":"835e5e3e-cda1-4cde-8e93-153ff2732d8a","Type":"ContainerStarted","Data":"a3786f573c6db70aadca7adc7a7bbde516d7bc7d89ac1bc0c81383c6859039db"}
Jul 15 19:54:31.341282 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.341250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerStarted","Data":"961d6a2f824400ed0297f134e5644700fffcde2a29130b1c69b5abc48f2f6226"}
Jul 15 19:54:31.353165 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.352947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ljpsf" event={"ID":"bd71fa81-6083-41ca-b02a-1b925656891b","Type":"ContainerStarted","Data":"874b5090c55543357bf689d6999760528364949caf8eb92df7db83c2ace2e575"}
Jul 15 19:54:31.363015 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.362984    2563 generic.go:356] "Generic (PLEG): container finished" podID="a42adba9814a4962e53f94f46a160544" containerID="3b6532d374b5187e2049eded14d7951b25d9fece4918c29a1494f8e91b3c4a6c" exitCode=0
Jul 15 19:54:31.363553 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.363509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" event={"ID":"a42adba9814a4962e53f94f46a160544","Type":"ContainerDied","Data":"3b6532d374b5187e2049eded14d7951b25d9fece4918c29a1494f8e91b3c4a6c"}
Jul 15 19:54:31.373774 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.373750    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn" event={"ID":"d7124dc7-022e-424a-8dfd-e4b1b22b5044","Type":"ContainerStarted","Data":"638211cc387939c49997c6c8794e40b2e8ceee9cd6a79101fab735eb6d5500c2"}
Jul 15 19:54:31.383115 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.383089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dhkth" event={"ID":"e0728008-4dd7-4bb5-8220-1679c4f82a4a","Type":"ContainerStarted","Data":"34da2426f8e21ae7476a46d12ccdcc02a36bc2877eef3e8f612c4fdaf09d38ca"}
Jul 15 19:54:31.404396 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.403595    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"95d6e977449761f2eccbf49b1af7b06efe6109ffb89588a93f0af89358d9d393"}
Jul 15 19:54:31.431232 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:31.431156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bs75x" event={"ID":"bf9871d5-3f5b-4020-a852-1a0ff24d99a3","Type":"ContainerStarted","Data":"6c859b4bc2f5c3fd581b7737bfee046286ded31102eb1cdfb8a136611ebc6c66"}
Jul 15 19:54:32.291920 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:32.291886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:32.292362 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.292014    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:32.442892 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:32.442857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" event={"ID":"a42adba9814a4962e53f94f46a160544","Type":"ContainerStarted","Data":"4e9e343d2b53e46bf6c9af33e7f884d480369f914be0b3d6e4ba582dc8515bd3"}
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:32.867628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:32.867740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:32.867805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.867965    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.867984    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.867997    2563 projected.go:196] Error preparing data for projected volume kube-api-access-ldmbd for pod openshift-network-diagnostics/network-check-target-hzrbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.868083    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd podName:1b9b5d7f-bd90-4818-9997-193de9a8eb7c nodeName:}" failed. No retries permitted until 2026-07-15 19:54:36.868065455 +0000 UTC m=+10.125467616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmbd" (UniqueName: "kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd") pod "network-check-target-hzrbl" (UID: "1b9b5d7f-bd90-4818-9997-193de9a8eb7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.868515    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.868575    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret podName:95419d71-9a42-48ac-a319-70c556aa50bf nodeName:}" failed. No retries permitted until 2026-07-15 19:54:36.868551195 +0000 UTC m=+10.125953353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret") pod "global-pull-secret-syncer-6qndr" (UID: "95419d71-9a42-48ac-a319-70c556aa50bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.868635    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:32.868702 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:32.868671    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:36.868658763 +0000 UTC m=+10.126060923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:33.291904 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:33.291823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:33.292052 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:33.291955    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:33.292425 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:33.291823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:33.292425 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:33.292407    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:34.291977 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:34.291945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:34.292161 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:34.292074    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:35.291700 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:35.291668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:35.291874 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:35.291816    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:35.292195 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:35.292164    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:35.292569 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:35.292261    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:36.291770 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:36.291738    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:36.291945 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.291844    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:36.904177 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:36.904141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:36.904177 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:36.904180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:36.904218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904344    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904415    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret podName:95419d71-9a42-48ac-a319-70c556aa50bf nodeName:}" failed. No retries permitted until 2026-07-15 19:54:44.904388484 +0000 UTC m=+18.161790666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret") pod "global-pull-secret-syncer-6qndr" (UID: "95419d71-9a42-48ac-a319-70c556aa50bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904495    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904531    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:44.904520135 +0000 UTC m=+18.161922306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904633    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904646    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:54:36.904681 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904661    2563 projected.go:196] Error preparing data for projected volume kube-api-access-ldmbd for pod openshift-network-diagnostics/network-check-target-hzrbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:36.905056 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:36.904695    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd podName:1b9b5d7f-bd90-4818-9997-193de9a8eb7c nodeName:}" failed. No retries permitted until 2026-07-15 19:54:44.904686989 +0000 UTC m=+18.162089145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmbd" (UniqueName: "kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd") pod "network-check-target-hzrbl" (UID: "1b9b5d7f-bd90-4818-9997-193de9a8eb7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:37.292132 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:37.292053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:37.292294 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:37.292170    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:37.292353 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:37.292304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:37.292437 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:37.292416    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:38.292133 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:38.292103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:38.292582 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:38.292213    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:39.296228 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:39.296195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:39.296653 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:39.296316    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:39.296738 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:39.296689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:39.296825 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:39.296805    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:40.291546 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:40.291516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:40.291740 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:40.291621    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:41.292416 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:41.292371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:41.292850 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:41.292517    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:41.292850 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:41.292572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:41.292850 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:41.292691    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:42.291993 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:42.291960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:42.292176 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:42.292073    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:43.291910 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:43.291876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:43.292417 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:43.292007    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:43.292417 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:43.292101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:43.292417 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:43.292211    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:44.292213 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:44.292178    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:44.292688 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.292294    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:44.966812 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:44.966775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:44.966987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:44.966852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:44.966987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:44.966872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:44.966987 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.966954    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:44.966987 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.966970    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:54:44.966987 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.966987    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:54:44.967200 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.966999    2563 projected.go:196] Error preparing data for projected volume kube-api-access-ldmbd for pod openshift-network-diagnostics/network-check-target-hzrbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:44.967200 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.966953    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:44.967200 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.967032    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret podName:95419d71-9a42-48ac-a319-70c556aa50bf nodeName:}" failed. No retries permitted until 2026-07-15 19:55:00.967016785 +0000 UTC m=+34.224418942 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret") pod "global-pull-secret-syncer-6qndr" (UID: "95419d71-9a42-48ac-a319-70c556aa50bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:54:44.967200 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.967074    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd podName:1b9b5d7f-bd90-4818-9997-193de9a8eb7c nodeName:}" failed. No retries permitted until 2026-07-15 19:55:00.967057025 +0000 UTC m=+34.224459185 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmbd" (UniqueName: "kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd") pod "network-check-target-hzrbl" (UID: "1b9b5d7f-bd90-4818-9997-193de9a8eb7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:54:44.967200 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:44.967096    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:00.967086844 +0000 UTC m=+34.224489007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:54:45.292521 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:45.292442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:45.292521 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:45.292489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:45.293078 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:45.292569    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:45.293078 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:45.292635    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:46.292426 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:46.292384    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:46.292587 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:46.292514    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:47.292941 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:47.292900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:47.293384 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:47.293008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:47.293384 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:47.293121    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:47.293384 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:47.293009    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:48.291450 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:48.291416    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:48.291620 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:48.291516    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:49.298802 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.293676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:49.298802 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:49.293825    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:49.298802 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.293863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:49.298802 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:49.293955    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:49.480335 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.480106    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn" event={"ID":"d7124dc7-022e-424a-8dfd-e4b1b22b5044","Type":"ContainerStarted","Data":"bea5f77f94007c8b2261f2cd7ffa4c428e8c302ed4fcd5a3fb19ea0d4816d36d"}
Jul 15 19:54:49.481423 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.481401    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dhkth" event={"ID":"e0728008-4dd7-4bb5-8220-1679c4f82a4a","Type":"ContainerStarted","Data":"9e682cf161b03fa695c5134e1484b64652c3fe325495f06c10371a8e4979d4d8"}
Jul 15 19:54:49.483004 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.482978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"5bdd313c152c7d96d09a3d4e7e071070a16a805d01e1d3adf4ea9fd4e4a4772f"}
Jul 15 19:54:49.484426 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.484402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bs75x" event={"ID":"bf9871d5-3f5b-4020-a852-1a0ff24d99a3","Type":"ContainerStarted","Data":"cbc8133725917e17c8e18584db7e589a8211ad14fc9094122b38375c00971bb4"}
Jul 15 19:54:49.486062 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.485986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nkvfh" event={"ID":"18c22562-fc3d-4b9c-aa81-02d56e4775c8","Type":"ContainerStarted","Data":"ba2204a4a6250739c583f50f9153a2224ec39be4075c6161b54522e1be37875d"}
Jul 15 19:54:49.488103 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.488081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkvvl" event={"ID":"835e5e3e-cda1-4cde-8e93-153ff2732d8a","Type":"ContainerStarted","Data":"86eb765bd77874b68f92810b8343882c08a836223122f65e73b60eb9f7336387"}
Jul 15 19:54:49.490006 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.489981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerStarted","Data":"174d603a9021bc2f660f2e2e1ee8fcc7c1289189ac460c3a3ab0ffe453886ba9"}
Jul 15 19:54:49.491548 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.491528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ljpsf" event={"ID":"bd71fa81-6083-41ca-b02a-1b925656891b","Type":"ContainerStarted","Data":"3e905dce3d1278a1f4e8d88298435f332f52ed27a90a1b15fe4b1f9ff3a1743d"}
Jul 15 19:54:49.499059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.499023    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" podStartSLOduration=21.499012341 podStartE2EDuration="21.499012341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:32.476942033 +0000 UTC m=+5.734344215" watchObservedRunningTime="2026-07-15 19:54:49.499012341 +0000 UTC m=+22.756414519"
Jul 15 19:54:49.499411 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.499384    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dhkth" podStartSLOduration=3.690642 podStartE2EDuration="22.499377912s" totalImagesPullingTime="18.808735912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:49.498960688 +0000 UTC m=+22.756362868" watchObservedRunningTime="2026-07-15 19:54:49.499377912 +0000 UTC m=+22.756780091"
Jul 15 19:54:49.552821 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.552763    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xkvvl" podStartSLOduration=3.791104653 podStartE2EDuration="22.55274094s" totalImagesPullingTime="18.761636287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:49.521340829 +0000 UTC m=+22.778743008" watchObservedRunningTime="2026-07-15 19:54:49.55274094 +0000 UTC m=+22.810143112"
Jul 15 19:54:49.552941 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.552899    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bs75x" podStartSLOduration=3.720360589 podStartE2EDuration="22.552890842s" totalImagesPullingTime="18.832530253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:49.552828667 +0000 UTC m=+22.810230845" watchObservedRunningTime="2026-07-15 19:54:49.552890842 +0000 UTC m=+22.810293021"
Jul 15 19:54:49.621775 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.621712    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nkvfh" podStartSLOduration=3.780365508 podStartE2EDuration="22.621696555s" totalImagesPullingTime="18.841331047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:49.620865141 +0000 UTC m=+22.878267321" watchObservedRunningTime="2026-07-15 19:54:49.621696555 +0000 UTC m=+22.879098756"
Jul 15 19:54:49.649311 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:49.649265    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ljpsf" podStartSLOduration=3.818385029 podStartE2EDuration="22.649249655s" totalImagesPullingTime="18.830864626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:49.649209605 +0000 UTC m=+22.906611786" watchObservedRunningTime="2026-07-15 19:54:49.649249655 +0000 UTC m=+22.906651833"
Jul 15 19:54:50.292133 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.291936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:50.292208 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:50.292189    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:50.394970 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.394948    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 19:54:50.496421 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.496331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c742j" event={"ID":"8a9a4ff3-7833-4ced-bbdd-504024118cb4","Type":"ContainerStarted","Data":"48300b0a4b628fa172a258c7065eabd542fac5a0bea51db310bba00ff6d77ac0"}
Jul 15 19:54:50.497877 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.497850    2563 generic.go:356] "Generic (PLEG): container finished" podID="ebf623c0-004d-49f5-81f2-1855e3aba827" containerID="174d603a9021bc2f660f2e2e1ee8fcc7c1289189ac460c3a3ab0ffe453886ba9" exitCode=0
Jul 15 19:54:50.497992 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.497928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerDied","Data":"174d603a9021bc2f660f2e2e1ee8fcc7c1289189ac460c3a3ab0ffe453886ba9"}
Jul 15 19:54:50.499783 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.499756    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn" event={"ID":"d7124dc7-022e-424a-8dfd-e4b1b22b5044","Type":"ContainerStarted","Data":"ddc58db14cf5b288ceb8c19e80f98b18e59165d437c9e0decb2dc295c00063c6"}
Jul 15 19:54:50.502647 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.502627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 19:54:50.503029 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.503006    2563 generic.go:356] "Generic (PLEG): container finished" podID="42abef4d-d44e-48a7-8847-904feec67e60" containerID="ba302a921a61181a1cbb589fdc3880fb91d699c3dc7accb16d8a839fd2b120b1" exitCode=1
Jul 15 19:54:50.503114 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.503036    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"bd3ae436f9f60ea938fe18e47512cc56f137b381e75f4d82bf4243bba96ef1e4"}
Jul 15 19:54:50.503114 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.503093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"f483fed68e793819e649a47523172cc69bec96dcb0ad35f608435ec7f4b45086"}
Jul 15 19:54:50.503194 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.503137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"02b298a1603e201a50ea2ba44170e08aab73df17cd49f23d2ef5aa9deb78b790"}
Jul 15 19:54:50.503194 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.503170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"353223f23e8562fb0983319f4e5e7e5872018ba0ab3179a2418abea5256ad096"}
Jul 15 19:54:50.503253 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.503202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerDied","Data":"ba302a921a61181a1cbb589fdc3880fb91d699c3dc7accb16d8a839fd2b120b1"}
Jul 15 19:54:50.514258 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:50.514222    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c742j" podStartSLOduration=4.710161973 podStartE2EDuration="23.51420977s" totalImagesPullingTime="18.804047797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:50.513705747 +0000 UTC m=+23.771107926" watchObservedRunningTime="2026-07-15 19:54:50.51420977 +0000 UTC m=+23.771611948"
Jul 15 19:54:51.176569 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:51.176362    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:54:50.394964077Z","UUID":"5402df7a-7fc8-4c56-ae9c-c31447acf788","Handler":null,"Name":"","Endpoint":""}
Jul 15 19:54:51.178522 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:51.178493    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 19:54:51.178656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:51.178536    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 19:54:51.293843 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:51.293819    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:51.293843 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:51.293837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:51.294051 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:51.293911    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:51.294051 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:51.294008    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:51.507954 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:51.507869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn" event={"ID":"d7124dc7-022e-424a-8dfd-e4b1b22b5044","Type":"ContainerStarted","Data":"51e4a877965d3cca2200549cffcea1db8c83eb2f352919cfe25d9864dda16436"}
Jul 15 19:54:51.533132 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:51.533079    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zwxrn" podStartSLOduration=4.001678385 podStartE2EDuration="24.533059097s" totalImagesPullingTime="20.531380712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:51.532949516 +0000 UTC m=+24.790351696" watchObservedRunningTime="2026-07-15 19:54:51.533059097 +0000 UTC m=+24.790461276"
Jul 15 19:54:52.291730 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:52.291638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:52.291892 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:52.291773    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:52.513204 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:52.513175    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 19:54:52.513975 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:52.513945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"3130d269f85e651ceeab55898f4f678c4763a0fb6cc26e91d6a330f436d9ddb1"}
Jul 15 19:54:52.684024 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:52.683944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:52.684560 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:52.684540    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:53.292383 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:53.292145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:53.292580 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:53.292199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:53.292580 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:53.292487    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:53.292580 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:53.292530    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:53.516315 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:53.516279    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:53.516779 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:53.516762    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xkvvl"
Jul 15 19:54:54.292080 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.292048    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:54.292245 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:54.292166    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:54.523268 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.523182    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 19:54:54.525110 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.524460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"39bcf3bb8fbac4767dceba4abb66c052f4124688d2283a23047d8238cd38ea3a"}
Jul 15 19:54:54.525110 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.524542    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:54.525110 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.524582    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:54.525110 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.524626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:54.525110 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.524681    2563 scope.go:117] "RemoveContainer" containerID="ba302a921a61181a1cbb589fdc3880fb91d699c3dc7accb16d8a839fd2b120b1"
Jul 15 19:54:54.543441 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.543340    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:54.544476 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:54.544336    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:54:55.291996 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:55.291806    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:55.292148 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:55.291866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:55.292148 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:55.292101    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:55.292148 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:55.292137    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:55.527220 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:55.527184    2563 generic.go:356] "Generic (PLEG): container finished" podID="ebf623c0-004d-49f5-81f2-1855e3aba827" containerID="ecb1463bbb2884557f63457e4d9005f1397509db621f84d8cedf015be927e505" exitCode=0
Jul 15 19:54:55.527679 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:55.527247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerDied","Data":"ecb1463bbb2884557f63457e4d9005f1397509db621f84d8cedf015be927e505"}
Jul 15 19:54:55.531008 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:55.530994    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 19:54:55.531329 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:55.531308    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" event={"ID":"42abef4d-d44e-48a7-8847-904feec67e60","Type":"ContainerStarted","Data":"414984bd0a9e7776babc33e3e7b9be75bcdc0c3ea98b4ab3825dbd0f03988b7d"}
Jul 15 19:54:55.667449 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:55.667347    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd" podStartSLOduration=9.773228461 podStartE2EDuration="28.667332498s" totalImagesPullingTime="18.894104037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:55.665456951 +0000 UTC m=+28.922859141" watchObservedRunningTime="2026-07-15 19:54:55.667332498 +0000 UTC m=+28.924734675"
Jul 15 19:54:56.292335 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:56.292308    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:56.292503 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:56.292416    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:56.535836 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:56.535756    2563 generic.go:356] "Generic (PLEG): container finished" podID="ebf623c0-004d-49f5-81f2-1855e3aba827" containerID="2beb9f8b4765283c75dc6aeea6fef89874ae98c5dafbaaccb6432c6bf8f31823" exitCode=0
Jul 15 19:54:56.536224 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:56.535840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerDied","Data":"2beb9f8b4765283c75dc6aeea6fef89874ae98c5dafbaaccb6432c6bf8f31823"}
Jul 15 19:54:57.014743 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.014640    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6qndr"]
Jul 15 19:54:57.014905 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.014782    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:57.014905 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:57.014879    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:54:57.024766 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.024739    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kc8fr"]
Jul 15 19:54:57.024887 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.024842    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:57.024955 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:57.024937    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:57.034295 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.034262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzrbl"]
Jul 15 19:54:57.034424 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.034345    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:57.034424 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:57.034416    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:57.540009 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.539816    2563 generic.go:356] "Generic (PLEG): container finished" podID="ebf623c0-004d-49f5-81f2-1855e3aba827" containerID="cd7070756216d837acdf76cff3b4567179b4a09eaa093489864f62145071056e" exitCode=0
Jul 15 19:54:57.540430 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:57.539896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerDied","Data":"cd7070756216d837acdf76cff3b4567179b4a09eaa093489864f62145071056e"}
Jul 15 19:54:58.291458 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:58.291432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:54:58.291595 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:58.291546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:54:58.291595 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:58.291556    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:54:58.291595 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:54:58.291585    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:54:58.291710 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:58.291690    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:54:58.291821 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:54:58.291797    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:55:00.291679 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:00.291632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:55:00.291679 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:00.291665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:55:00.292406 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:00.291632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:55:00.292406 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.291773    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzrbl" podUID="1b9b5d7f-bd90-4818-9997-193de9a8eb7c"
Jul 15 19:55:00.292406 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.291853    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qndr" podUID="95419d71-9a42-48ac-a319-70c556aa50bf"
Jul 15 19:55:00.292406 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.291947    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kc8fr" podUID="760b10f9-2f25-4197-9678-518c2f396178"
Jul 15 19:55:00.986531 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:00.986444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:55:00.986531 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:00.986488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986598    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986610    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:00.986619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986634    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986672    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:32.986651383 +0000 UTC m=+66.244053555 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986679    2563 projected.go:196] Error preparing data for projected volume kube-api-access-ldmbd for pod openshift-network-diagnostics/network-check-target-hzrbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986682    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986746    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd podName:1b9b5d7f-bd90-4818-9997-193de9a8eb7c nodeName:}" failed. No retries permitted until 2026-07-15 19:55:32.98671294 +0000 UTC m=+66.244115098 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmbd" (UniqueName: "kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd") pod "network-check-target-hzrbl" (UID: "1b9b5d7f-bd90-4818-9997-193de9a8eb7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:55:00.986920 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:00.986766    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret podName:95419d71-9a42-48ac-a319-70c556aa50bf nodeName:}" failed. No retries permitted until 2026-07-15 19:55:32.986756321 +0000 UTC m=+66.244158477 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret") pod "global-pull-secret-syncer-6qndr" (UID: "95419d71-9a42-48ac-a319-70c556aa50bf") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:55:02.048694 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.048644    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeReady"
Jul 15 19:55:02.049185 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.048803    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 19:55:02.147443 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.147416    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cwl2l"]
Jul 15 19:55:02.167377 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.167349    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-874vx"]
Jul 15 19:55:02.167489 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.167446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.171611 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.171587    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 19:55:02.171770 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.171629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c5qnq\""
Jul 15 19:55:02.175099 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.175027    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 19:55:02.187583 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.187563    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwl2l"]
Jul 15 19:55:02.187740 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.187604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-874vx"]
Jul 15 19:55:02.187740 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.187678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.193196 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.193173    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 19:55:02.193320 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.193210    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zbcvf\""
Jul 15 19:55:02.193320 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.193184    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 19:55:02.193787 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.193767    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 19:55:02.291910 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.291886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:55:02.292029 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.291985    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:55:02.292217 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.292198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:55:02.294754 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.294738    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:55:02.297362 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm9rf\" (UniqueName: \"kubernetes.io/projected/42abe83f-a11e-466c-a4b5-c7a3a0c2a83f-kube-api-access-bm9rf\") pod \"ingress-canary-874vx\" (UID: \"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f\") " pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.297455 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd40f713-e53b-4ddd-806b-298eda2be912-config-volume\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.297455 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlcw6\" (UniqueName: \"kubernetes.io/projected/dd40f713-e53b-4ddd-806b-298eda2be912-kube-api-access-rlcw6\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.297455 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.297707 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40f713-e53b-4ddd-806b-298eda2be912-tmp-dir\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.297707 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42abe83f-a11e-466c-a4b5-c7a3a0c2a83f-cert\") pod \"ingress-canary-874vx\" (UID: \"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f\") " pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.297707 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjg99\""
Jul 15 19:55:02.297872 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.297776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:55:02.298215 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.298197    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:55:02.298345 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.298329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:55:02.304445 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.304430    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q7pb7\""
Jul 15 19:55:02.398141 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlcw6\" (UniqueName: \"kubernetes.io/projected/dd40f713-e53b-4ddd-806b-298eda2be912-kube-api-access-rlcw6\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.398237 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.398237 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40f713-e53b-4ddd-806b-298eda2be912-tmp-dir\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.398303 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42abe83f-a11e-466c-a4b5-c7a3a0c2a83f-cert\") pod \"ingress-canary-874vx\" (UID: \"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f\") " pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.398303 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:02.398256    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:55:02.398303 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm9rf\" (UniqueName: \"kubernetes.io/projected/42abe83f-a11e-466c-a4b5-c7a3a0c2a83f-kube-api-access-bm9rf\") pod \"ingress-canary-874vx\" (UID: \"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f\") " pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.398303 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:02.398298    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls podName:dd40f713-e53b-4ddd-806b-298eda2be912 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:02.898285126 +0000 UTC m=+36.155687282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls") pod "dns-default-cwl2l" (UID: "dd40f713-e53b-4ddd-806b-298eda2be912") : secret "dns-default-metrics-tls" not found
Jul 15 19:55:02.398466 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd40f713-e53b-4ddd-806b-298eda2be912-config-volume\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.398640 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40f713-e53b-4ddd-806b-298eda2be912-tmp-dir\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.398811 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.398796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd40f713-e53b-4ddd-806b-298eda2be912-config-volume\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.402524 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.402505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42abe83f-a11e-466c-a4b5-c7a3a0c2a83f-cert\") pod \"ingress-canary-874vx\" (UID: \"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f\") " pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.414473 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.414456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm9rf\" (UniqueName: \"kubernetes.io/projected/42abe83f-a11e-466c-a4b5-c7a3a0c2a83f-kube-api-access-bm9rf\") pod \"ingress-canary-874vx\" (UID: \"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f\") " pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.415885 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.415866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlcw6\" (UniqueName: \"kubernetes.io/projected/dd40f713-e53b-4ddd-806b-298eda2be912-kube-api-access-rlcw6\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.496445 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.496419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-874vx"
Jul 15 19:55:02.554252 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.554187    2563 generic.go:356] "Generic (PLEG): container finished" podID="ebf623c0-004d-49f5-81f2-1855e3aba827" containerID="c46513c0bd1e6c9319672f8d83833cf7873123255017ab3211e3d424a214104a" exitCode=0
Jul 15 19:55:02.554252 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.554248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerDied","Data":"c46513c0bd1e6c9319672f8d83833cf7873123255017ab3211e3d424a214104a"}
Jul 15 19:55:02.725299 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.725114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-874vx"]
Jul 15 19:55:02.728652 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:55:02.728630    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42abe83f_a11e_466c_a4b5_c7a3a0c2a83f.slice/crio-26542bd941d1ee38f09b58971e1162e8364751b73e6c652de3780e2e1bf03046 WatchSource:0}: Error finding container 26542bd941d1ee38f09b58971e1162e8364751b73e6c652de3780e2e1bf03046: Status 404 returned error can't find the container with id 26542bd941d1ee38f09b58971e1162e8364751b73e6c652de3780e2e1bf03046
Jul 15 19:55:02.902083 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:02.901969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:02.902083 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:02.902083    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:55:02.902263 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:02.902135    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls podName:dd40f713-e53b-4ddd-806b-298eda2be912 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:03.902122291 +0000 UTC m=+37.159524447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls") pod "dns-default-cwl2l" (UID: "dd40f713-e53b-4ddd-806b-298eda2be912") : secret "dns-default-metrics-tls" not found
Jul 15 19:55:03.560420 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:03.560382    2563 generic.go:356] "Generic (PLEG): container finished" podID="ebf623c0-004d-49f5-81f2-1855e3aba827" containerID="73889716e684a382473dedc79ad7975007d9b3c2e26df44bc208a5b9f3e65da7" exitCode=0
Jul 15 19:55:03.561021 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:03.560466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerDied","Data":"73889716e684a382473dedc79ad7975007d9b3c2e26df44bc208a5b9f3e65da7"}
Jul 15 19:55:03.561713 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:03.561688    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-874vx" event={"ID":"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f","Type":"ContainerStarted","Data":"26542bd941d1ee38f09b58971e1162e8364751b73e6c652de3780e2e1bf03046"}
Jul 15 19:55:03.910429 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:03.910336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:03.910597 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:03.910516    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:55:03.910597 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:03.910584    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls podName:dd40f713-e53b-4ddd-806b-298eda2be912 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:05.91056522 +0000 UTC m=+39.167967380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls") pod "dns-default-cwl2l" (UID: "dd40f713-e53b-4ddd-806b-298eda2be912") : secret "dns-default-metrics-tls" not found
Jul 15 19:55:04.568432 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:04.568395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zttrk" event={"ID":"ebf623c0-004d-49f5-81f2-1855e3aba827","Type":"ContainerStarted","Data":"b88af674164ea51c42e4b1d07a3b6fe1de0e8812a55d40bc6cfb9ca1890f721e"}
Jul 15 19:55:04.611249 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:04.611182    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zttrk" podStartSLOduration=9.16207776 podStartE2EDuration="37.611165871s" totalImagesPullingTime="28.449088111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:04.609375744 +0000 UTC m=+37.866777922" watchObservedRunningTime="2026-07-15 19:55:04.611165871 +0000 UTC m=+37.868568095"
Jul 15 19:55:05.572244 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:05.571929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-874vx" event={"ID":"42abe83f-a11e-466c-a4b5-c7a3a0c2a83f","Type":"ContainerStarted","Data":"add3359304a2acd96cc7245d481bbf10b1b2e9aff05acf2b5f30b86986805632"}
Jul 15 19:55:05.601158 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:05.601114    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-874vx" podStartSLOduration=1.151815094 podStartE2EDuration="3.601100679s" totalImagesPullingTime="2.449285585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:05.597540105 +0000 UTC m=+38.854942282" watchObservedRunningTime="2026-07-15 19:55:05.601100679 +0000 UTC m=+38.858502856"
Jul 15 19:55:05.924585 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:05.924547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:05.924746 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:05.924685    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:55:05.924807 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:05.924779    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls podName:dd40f713-e53b-4ddd-806b-298eda2be912 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:09.924761599 +0000 UTC m=+43.182163757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls") pod "dns-default-cwl2l" (UID: "dd40f713-e53b-4ddd-806b-298eda2be912") : secret "dns-default-metrics-tls" not found
Jul 15 19:55:09.952784 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:09.952742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:09.953256 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:09.952905    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:55:09.953256 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:09.952971    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls podName:dd40f713-e53b-4ddd-806b-298eda2be912 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:17.952953329 +0000 UTC m=+51.210355485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls") pod "dns-default-cwl2l" (UID: "dd40f713-e53b-4ddd-806b-298eda2be912") : secret "dns-default-metrics-tls" not found
Jul 15 19:55:18.004946 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:18.004903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:18.005507 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:18.005028    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:55:18.005507 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:18.005102    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls podName:dd40f713-e53b-4ddd-806b-298eda2be912 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:34.005081703 +0000 UTC m=+67.262483863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls") pod "dns-default-cwl2l" (UID: "dd40f713-e53b-4ddd-806b-298eda2be912") : secret "dns-default-metrics-tls" not found
Jul 15 19:55:26.551926 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:26.551901    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sfcd"
Jul 15 19:55:33.008492 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.008454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:55:33.008492 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.008493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:55:33.009040 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.008520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:55:33.013304 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.013284    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:55:33.013408 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.013287    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:55:33.013408 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.013287    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:55:33.019207 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:33.019183    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 19:55:33.019313 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:33.019266    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs podName:760b10f9-2f25-4197-9678-518c2f396178 nodeName:}" failed. No retries permitted until 2026-07-15 19:56:37.019245402 +0000 UTC m=+130.276647562 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs") pod "network-metrics-daemon-kc8fr" (UID: "760b10f9-2f25-4197-9678-518c2f396178") : secret "metrics-daemon-secret" not found
Jul 15 19:55:33.022902 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.022885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95419d71-9a42-48ac-a319-70c556aa50bf-original-pull-secret\") pod \"global-pull-secret-syncer-6qndr\" (UID: \"95419d71-9a42-48ac-a319-70c556aa50bf\") " pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:55:33.023856 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.023844    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:55:33.031423 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.031406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldmbd\" (UniqueName: \"kubernetes.io/projected/1b9b5d7f-bd90-4818-9997-193de9a8eb7c-kube-api-access-ldmbd\") pod \"network-check-target-hzrbl\" (UID: \"1b9b5d7f-bd90-4818-9997-193de9a8eb7c\") " pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:55:33.205816 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.205791    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qndr"
Jul 15 19:55:33.222299 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.222281    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q7pb7\""
Jul 15 19:55:33.228235 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.228215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:55:33.389036 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.389005    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6qndr"]
Jul 15 19:55:33.392962 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:55:33.392933    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95419d71_9a42_48ac_a319_70c556aa50bf.slice/crio-78b27465eac593b8b642d611316079605e979c02577b98c27ad06bb7c6987e97 WatchSource:0}: Error finding container 78b27465eac593b8b642d611316079605e979c02577b98c27ad06bb7c6987e97: Status 404 returned error can't find the container with id 78b27465eac593b8b642d611316079605e979c02577b98c27ad06bb7c6987e97
Jul 15 19:55:33.429146 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.429120    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzrbl"]
Jul 15 19:55:33.636909 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.636875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6qndr" event={"ID":"95419d71-9a42-48ac-a319-70c556aa50bf","Type":"ContainerStarted","Data":"78b27465eac593b8b642d611316079605e979c02577b98c27ad06bb7c6987e97"}
Jul 15 19:55:33.637841 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:33.637819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzrbl" event={"ID":"1b9b5d7f-bd90-4818-9997-193de9a8eb7c","Type":"ContainerStarted","Data":"263412738ff9069c0890bf7aacd13c8140303881d512a55fbeeb802233f988ce"}
Jul 15 19:55:34.014667 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.014590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:55:34.015187 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:34.014760    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:55:34.015187 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:34.014821    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls podName:dd40f713-e53b-4ddd-806b-298eda2be912 nodeName:}" failed. No retries permitted until 2026-07-15 19:56:06.014804141 +0000 UTC m=+99.272206306 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls") pod "dns-default-cwl2l" (UID: "dd40f713-e53b-4ddd-806b-298eda2be912") : secret "dns-default-metrics-tls" not found
Jul 15 19:55:34.134677 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.134639    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"]
Jul 15 19:55:34.136607 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.136584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"
Jul 15 19:55:34.144649 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.144336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:55:34.144990 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.144968    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 15 19:55:34.145094 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.145022    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-dtl64\""
Jul 15 19:55:34.167383 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.167359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"]
Jul 15 19:55:34.217213 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.217184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcs9p\" (UniqueName: \"kubernetes.io/projected/b235eecb-c48b-4c23-919d-502ca9aece5d-kube-api-access-fcs9p\") pod \"volume-data-source-validator-78495597f6-vdxvd\" (UID: \"b235eecb-c48b-4c23-919d-502ca9aece5d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"
Jul 15 19:55:34.318049 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.317958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcs9p\" (UniqueName: \"kubernetes.io/projected/b235eecb-c48b-4c23-919d-502ca9aece5d-kube-api-access-fcs9p\") pod \"volume-data-source-validator-78495597f6-vdxvd\" (UID: \"b235eecb-c48b-4c23-919d-502ca9aece5d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"
Jul 15 19:55:34.356411 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.356371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcs9p\" (UniqueName: \"kubernetes.io/projected/b235eecb-c48b-4c23-919d-502ca9aece5d-kube-api-access-fcs9p\") pod \"volume-data-source-validator-78495597f6-vdxvd\" (UID: \"b235eecb-c48b-4c23-919d-502ca9aece5d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"
Jul 15 19:55:34.360768 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.360743    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-pqfl4"]
Jul 15 19:55:34.362605 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.362587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.371911 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.371885    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 15 19:55:34.372258 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.372238    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-nm75h\""
Jul 15 19:55:34.373272 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.373253    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 15 19:55:34.373377 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.373255    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 15 19:55:34.374681 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.374658    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79c74b977d-stx57"]
Jul 15 19:55:34.376455 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.376439    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.405434 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.405267    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 15 19:55:34.405434 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.405280    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:55:34.405434 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.405306    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 15 19:55:34.405434 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.405306    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 15 19:55:34.405434 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.405402    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 19:55:34.405732 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.405510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 15 19:55:34.406065 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.406041    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 19:55:34.406168 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.406080    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-dnkb5\""
Jul 15 19:55:34.408195 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.408175    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 15 19:55:34.418294 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.418414 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05d1c048-b150-4468-a377-1390dbe7a1ad-serving-cert\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.418414 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8scwq\" (UniqueName: \"kubernetes.io/projected/05d1c048-b150-4468-a377-1390dbe7a1ad-kube-api-access-8scwq\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.418414 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.418414 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05d1c048-b150-4468-a377-1390dbe7a1ad-trusted-ca\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.418603 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l2b6\" (UniqueName: \"kubernetes.io/projected/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-kube-api-access-9l2b6\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.418603 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-default-certificate\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.418603 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-stats-auth\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.418603 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.418506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05d1c048-b150-4468-a377-1390dbe7a1ad-config\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.429294 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.429275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79c74b977d-stx57"]
Jul 15 19:55:34.431123 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.431101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-pqfl4"]
Jul 15 19:55:34.447782 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.447750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"
Jul 15 19:55:34.519302 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05d1c048-b150-4468-a377-1390dbe7a1ad-trusted-ca\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.519453 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l2b6\" (UniqueName: \"kubernetes.io/projected/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-kube-api-access-9l2b6\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.519453 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-default-certificate\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.519453 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-stats-auth\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.519453 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05d1c048-b150-4468-a377-1390dbe7a1ad-config\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.519657 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.519861 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05d1c048-b150-4468-a377-1390dbe7a1ad-serving-cert\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.519861 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8scwq\" (UniqueName: \"kubernetes.io/projected/05d1c048-b150-4468-a377-1390dbe7a1ad-kube-api-access-8scwq\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.519861 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.519783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.520242 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:34.519884    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:35.01986192 +0000 UTC m=+68.277264080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : configmap references non-existent config key: service-ca.crt
Jul 15 19:55:34.520242 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:34.519883    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:55:34.520242 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:34.519937    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:35.01992431 +0000 UTC m=+68.277326470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : secret "router-metrics-certs-default" not found
Jul 15 19:55:34.520242 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.520180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05d1c048-b150-4468-a377-1390dbe7a1ad-config\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.520482 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.520305    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05d1c048-b150-4468-a377-1390dbe7a1ad-trusted-ca\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.522383 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.522358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-default-certificate\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.522732 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.522699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05d1c048-b150-4468-a377-1390dbe7a1ad-serving-cert\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.522882 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.522850    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-stats-auth\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.553067 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.553037    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8scwq\" (UniqueName: \"kubernetes.io/projected/05d1c048-b150-4468-a377-1390dbe7a1ad-kube-api-access-8scwq\") pod \"console-operator-7866497494-pqfl4\" (UID: \"05d1c048-b150-4468-a377-1390dbe7a1ad\") " pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.565489 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.565462    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l2b6\" (UniqueName: \"kubernetes.io/projected/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-kube-api-access-9l2b6\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:34.673087 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.673051    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:34.693246 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.693216    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd"]
Jul 15 19:55:34.697121 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:55:34.697094    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb235eecb_c48b_4c23_919d_502ca9aece5d.slice/crio-80260c491e0d325bd95b0e1b0dbdd0e1c8eed35866d7ebf78febf771d5303b7b WatchSource:0}: Error finding container 80260c491e0d325bd95b0e1b0dbdd0e1c8eed35866d7ebf78febf771d5303b7b: Status 404 returned error can't find the container with id 80260c491e0d325bd95b0e1b0dbdd0e1c8eed35866d7ebf78febf771d5303b7b
Jul 15 19:55:34.864985 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:34.864941    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-pqfl4"]
Jul 15 19:55:34.870370 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:55:34.870340    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d1c048_b150_4468_a377_1390dbe7a1ad.slice/crio-2ae457ea18301d3f94c76607a2accadf77d35bf7617c1744163529b7c2a6978a WatchSource:0}: Error finding container 2ae457ea18301d3f94c76607a2accadf77d35bf7617c1744163529b7c2a6978a: Status 404 returned error can't find the container with id 2ae457ea18301d3f94c76607a2accadf77d35bf7617c1744163529b7c2a6978a
Jul 15 19:55:35.022423 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:35.022349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:35.022423 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:35.022401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:35.022932 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:35.022519    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:55:35.022932 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:35.022583    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:36.022562414 +0000 UTC m=+69.279964575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : secret "router-metrics-certs-default" not found
Jul 15 19:55:35.023051 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:35.023024    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:36.023007813 +0000 UTC m=+69.280409972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : configmap references non-existent config key: service-ca.crt
Jul 15 19:55:35.645309 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:35.645264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pqfl4" event={"ID":"05d1c048-b150-4468-a377-1390dbe7a1ad","Type":"ContainerStarted","Data":"2ae457ea18301d3f94c76607a2accadf77d35bf7617c1744163529b7c2a6978a"}
Jul 15 19:55:35.646913 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:35.646878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd" event={"ID":"b235eecb-c48b-4c23-919d-502ca9aece5d","Type":"ContainerStarted","Data":"80260c491e0d325bd95b0e1b0dbdd0e1c8eed35866d7ebf78febf771d5303b7b"}
Jul 15 19:55:36.031247 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:36.031204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:36.031697 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:36.031264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:36.031697 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:36.031404    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:55:36.031697 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:36.031463    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:38.031444501 +0000 UTC m=+71.288846658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : secret "router-metrics-certs-default" not found
Jul 15 19:55:36.031697 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:36.031670    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:38.031654262 +0000 UTC m=+71.289056422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : configmap references non-existent config key: service-ca.crt
Jul 15 19:55:36.651508 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:36.651466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd" event={"ID":"b235eecb-c48b-4c23-919d-502ca9aece5d","Type":"ContainerStarted","Data":"946365a4c4b6e66b106cb9003ee3fcfe7c68866d17c785b2a44cdd309a3943c1"}
Jul 15 19:55:36.678478 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:36.678412    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vdxvd" podStartSLOduration=1.527109067 podStartE2EDuration="2.678392633s" totalImagesPullingTime="1.151283566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:36.677600976 +0000 UTC m=+69.935003157" watchObservedRunningTime="2026-07-15 19:55:36.678392633 +0000 UTC m=+69.935794797"
Jul 15 19:55:38.047692 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:38.047646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:38.048155 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:38.047772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:38.048155 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:38.047834    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:55:38.048155 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:38.047903    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:42.047884799 +0000 UTC m=+75.305286958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : configmap references non-existent config key: service-ca.crt
Jul 15 19:55:38.048155 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:38.047924    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:42.047913734 +0000 UTC m=+75.305315896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : secret "router-metrics-certs-default" not found
Jul 15 19:55:38.658740 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:38.658686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6qndr" event={"ID":"95419d71-9a42-48ac-a319-70c556aa50bf","Type":"ContainerStarted","Data":"ed6cee75911bad963764645089500a0a84dafaca3c4590b69aa966bd2767b723"}
Jul 15 19:55:38.660440 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:38.660409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzrbl" event={"ID":"1b9b5d7f-bd90-4818-9997-193de9a8eb7c","Type":"ContainerStarted","Data":"27bef85daab7f88eefc1942c60ece16ced1cd892bb1c958aa568207796e7847b"}
Jul 15 19:55:38.660581 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:38.660556    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:55:38.843625 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:38.843578    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6qndr" podStartSLOduration=66.955266236 podStartE2EDuration="1m11.84356392s" totalImagesPullingTime="4.888297684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:38.839403612 +0000 UTC m=+72.096805789" watchObservedRunningTime="2026-07-15 19:55:38.84356392 +0000 UTC m=+72.100966097"
Jul 15 19:55:38.900129 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:38.900078    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hzrbl" podStartSLOduration=67.05441654 podStartE2EDuration="1m11.90005958s" totalImagesPullingTime="4.84564304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:38.90003712 +0000 UTC m=+72.157439298" watchObservedRunningTime="2026-07-15 19:55:38.90005958 +0000 UTC m=+72.157461759"
Jul 15 19:55:40.449486 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.449454    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wttqf"]
Jul 15 19:55:40.452534 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.452517    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.460492 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.460469    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 19:55:40.460611 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.460569    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 19:55:40.460611 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.460586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 19:55:40.460611 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.460603    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 19:55:40.461362 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.461348    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cfjnb\""
Jul 15 19:55:40.472184 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.472165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wttqf"]
Jul 15 19:55:40.567107 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.567070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f69ca9b-1411-4a49-9a5d-e533e3529026-data-volume\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.567249 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.567120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f69ca9b-1411-4a49-9a5d-e533e3529026-crio-socket\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.567249 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.567140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.567249 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.567194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf5jm\" (UniqueName: \"kubernetes.io/projected/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-api-access-zf5jm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.567249 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.567234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.667560 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.667749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f69ca9b-1411-4a49-9a5d-e533e3529026-data-volume\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.667749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f69ca9b-1411-4a49-9a5d-e533e3529026-crio-socket\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.667749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.667749 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:40.667670    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:40.667749 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf5jm\" (UniqueName: \"kubernetes.io/projected/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-api-access-zf5jm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.667749 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:40.667750    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:41.167713377 +0000 UTC m=+74.425115536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:40.668059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667760    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/0.log"
Jul 15 19:55:40.668059 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:40.667773    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:55:40.668059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f69ca9b-1411-4a49-9a5d-e533e3529026-crio-socket\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.668059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667793    2563 generic.go:356] "Generic (PLEG): container finished" podID="05d1c048-b150-4468-a377-1390dbe7a1ad" containerID="fc7b23f41308a50bbfa7358dc357b7c2d4b8dea0521678dfa7e0eb9fcc0fd0a2" exitCode=255
Jul 15 19:55:40.668059 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:40.667810    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:41.167799526 +0000 UTC m=+74.425201682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : configmap "kube-rbac-proxy" not found
Jul 15 19:55:40.668059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pqfl4" event={"ID":"05d1c048-b150-4468-a377-1390dbe7a1ad","Type":"ContainerDied","Data":"fc7b23f41308a50bbfa7358dc357b7c2d4b8dea0521678dfa7e0eb9fcc0fd0a2"}
Jul 15 19:55:40.668059 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.667994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f69ca9b-1411-4a49-9a5d-e533e3529026-data-volume\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.668370 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.668171    2563 scope.go:117] "RemoveContainer" containerID="fc7b23f41308a50bbfa7358dc357b7c2d4b8dea0521678dfa7e0eb9fcc0fd0a2"
Jul 15 19:55:40.685831 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.685814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf5jm\" (UniqueName: \"kubernetes.io/projected/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-api-access-zf5jm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:40.883464 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.883430    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8f6sf"]
Jul 15 19:55:40.886213 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.886194    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:40.890238 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.890217    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 15 19:55:40.891363 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.891336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 15 19:55:40.891466 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.891370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 15 19:55:40.893678 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.893654    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 15 19:55:40.893785 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.893743    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cr4rw\""
Jul 15 19:55:40.904052 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.904033    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8f6sf"]
Jul 15 19:55:40.970285 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.970200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9d7e291d-9b82-4a92-af85-d5e8c26b6970-signing-key\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:40.970285 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.970240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9d7e291d-9b82-4a92-af85-d5e8c26b6970-signing-cabundle\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:40.970455 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:40.970327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dl72\" (UniqueName: \"kubernetes.io/projected/9d7e291d-9b82-4a92-af85-d5e8c26b6970-kube-api-access-7dl72\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.070975 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.070952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9d7e291d-9b82-4a92-af85-d5e8c26b6970-signing-key\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.071086 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.070993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9d7e291d-9b82-4a92-af85-d5e8c26b6970-signing-cabundle\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.071182 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.071167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dl72\" (UniqueName: \"kubernetes.io/projected/9d7e291d-9b82-4a92-af85-d5e8c26b6970-kube-api-access-7dl72\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.071588 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.071566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9d7e291d-9b82-4a92-af85-d5e8c26b6970-signing-cabundle\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.073427 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.073399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9d7e291d-9b82-4a92-af85-d5e8c26b6970-signing-key\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.085831 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.085810    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dl72\" (UniqueName: \"kubernetes.io/projected/9d7e291d-9b82-4a92-af85-d5e8c26b6970-kube-api-access-7dl72\") pod \"service-ca-79f68b9f4f-8f6sf\" (UID: \"9d7e291d-9b82-4a92-af85-d5e8c26b6970\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.171604 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.171569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:41.171764 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.171611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:41.171764 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:41.171699    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:55:41.171836 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:41.171770    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:41.171836 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:41.171777    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:42.171762984 +0000 UTC m=+75.429165140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : configmap "kube-rbac-proxy" not found
Jul 15 19:55:41.171836 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:41.171816    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:42.171805079 +0000 UTC m=+75.429207236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:41.194254 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.194232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf"
Jul 15 19:55:41.325217 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.325185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8f6sf"]
Jul 15 19:55:41.330120 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:55:41.330089    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d7e291d_9b82_4a92_af85_d5e8c26b6970.slice/crio-1eeab2429574b754828e6510e0c8d48fc7cb1df9bec72e66eb51b136204c3a57 WatchSource:0}: Error finding container 1eeab2429574b754828e6510e0c8d48fc7cb1df9bec72e66eb51b136204c3a57: Status 404 returned error can't find the container with id 1eeab2429574b754828e6510e0c8d48fc7cb1df9bec72e66eb51b136204c3a57
Jul 15 19:55:41.675739 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.675691    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/1.log"
Jul 15 19:55:41.676138 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.676124    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/0.log"
Jul 15 19:55:41.676193 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.676163    2563 generic.go:356] "Generic (PLEG): container finished" podID="05d1c048-b150-4468-a377-1390dbe7a1ad" containerID="18d611bfe94b318636acfa83e2c9f18781a53dcd8ae2dd11b993d07117fc5b2a" exitCode=255
Jul 15 19:55:41.676250 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.676234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pqfl4" event={"ID":"05d1c048-b150-4468-a377-1390dbe7a1ad","Type":"ContainerDied","Data":"18d611bfe94b318636acfa83e2c9f18781a53dcd8ae2dd11b993d07117fc5b2a"}
Jul 15 19:55:41.676351 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.676319    2563 scope.go:117] "RemoveContainer" containerID="fc7b23f41308a50bbfa7358dc357b7c2d4b8dea0521678dfa7e0eb9fcc0fd0a2"
Jul 15 19:55:41.676628 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.676611    2563 scope.go:117] "RemoveContainer" containerID="18d611bfe94b318636acfa83e2c9f18781a53dcd8ae2dd11b993d07117fc5b2a"
Jul 15 19:55:41.676839 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:41.676818    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-pqfl4_openshift-console-operator(05d1c048-b150-4468-a377-1390dbe7a1ad)\"" pod="openshift-console-operator/console-operator-7866497494-pqfl4" podUID="05d1c048-b150-4468-a377-1390dbe7a1ad"
Jul 15 19:55:41.677537 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:41.677518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf" event={"ID":"9d7e291d-9b82-4a92-af85-d5e8c26b6970","Type":"ContainerStarted","Data":"1eeab2429574b754828e6510e0c8d48fc7cb1df9bec72e66eb51b136204c3a57"}
Jul 15 19:55:42.077492 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:42.077413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:42.077652 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:42.077514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:42.077652 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.077568    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:55:42.077652 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.077633    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:50.077614351 +0000 UTC m=+83.335016518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : secret "router-metrics-certs-default" not found
Jul 15 19:55:42.077652 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.077649    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:50.07764206 +0000 UTC m=+83.335044219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : configmap references non-existent config key: service-ca.crt
Jul 15 19:55:42.178401 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:42.178360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:42.178573 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:42.178413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:42.178573 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.178473    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:55:42.178573 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.178539    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:44.178521276 +0000 UTC m=+77.435923434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : configmap "kube-rbac-proxy" not found
Jul 15 19:55:42.178716 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.178579    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:42.178716 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.178657    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:44.178638361 +0000 UTC m=+77.436040517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:42.681909 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:42.681878    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/1.log"
Jul 15 19:55:42.682264 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:42.682172    2563 scope.go:117] "RemoveContainer" containerID="18d611bfe94b318636acfa83e2c9f18781a53dcd8ae2dd11b993d07117fc5b2a"
Jul 15 19:55:42.682336 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:42.682317    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-pqfl4_openshift-console-operator(05d1c048-b150-4468-a377-1390dbe7a1ad)\"" pod="openshift-console-operator/console-operator-7866497494-pqfl4" podUID="05d1c048-b150-4468-a377-1390dbe7a1ad"
Jul 15 19:55:43.433567 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:43.433542    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ljpsf_bd71fa81-6083-41ca-b02a-1b925656891b/dns-node-resolver/0.log"
Jul 15 19:55:43.686732 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:43.686700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf" event={"ID":"9d7e291d-9b82-4a92-af85-d5e8c26b6970","Type":"ContainerStarted","Data":"77bcee721b88d0524d940152c34b0f11b34f95ca6272f426d253561352d025d2"}
Jul 15 19:55:43.708301 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:43.708251    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-8f6sf" podStartSLOduration=1.740970371 podStartE2EDuration="3.708231765s" totalImagesPullingTime="1.967261394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:43.707618269 +0000 UTC m=+76.965020447" watchObservedRunningTime="2026-07-15 19:55:43.708231765 +0000 UTC m=+76.965633945"
Jul 15 19:55:44.195298 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:44.195262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:44.195298 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:44.195302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:44.195537 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:44.195382    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:55:44.195537 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:44.195398    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:44.195537 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:44.195450    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:48.195435593 +0000 UTC m=+81.452837753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:44.195537 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:44.195463    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:48.195457263 +0000 UTC m=+81.452859419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : configmap "kube-rbac-proxy" not found
Jul 15 19:55:44.454474 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:44.454408    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dhkth_e0728008-4dd7-4bb5-8220-1679c4f82a4a/node-ca/0.log"
Jul 15 19:55:44.674253 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:44.673262    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:44.674253 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:44.673326    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:55:44.674253 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:44.673747    2563 scope.go:117] "RemoveContainer" containerID="18d611bfe94b318636acfa83e2c9f18781a53dcd8ae2dd11b993d07117fc5b2a"
Jul 15 19:55:44.674253 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:44.673919    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-pqfl4_openshift-console-operator(05d1c048-b150-4468-a377-1390dbe7a1ad)\"" pod="openshift-console-operator/console-operator-7866497494-pqfl4" podUID="05d1c048-b150-4468-a377-1390dbe7a1ad"
Jul 15 19:55:44.835628 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:44.835507    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-874vx_42abe83f-a11e-466c-a4b5-c7a3a0c2a83f/serve-healthcheck-canary/0.log"
Jul 15 19:55:48.226109 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:48.225969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:48.226109 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:48.226060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:48.226611 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:48.226119    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:55:48.226611 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:48.226134    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:48.226611 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:48.226179    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:56.226162538 +0000 UTC m=+89.483564694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : configmap "kube-rbac-proxy" not found
Jul 15 19:55:48.226611 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:48.226196    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls podName:3f69ca9b-1411-4a49-9a5d-e533e3529026 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:56.226186897 +0000 UTC m=+89.483589054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wttqf" (UID: "3f69ca9b-1411-4a49-9a5d-e533e3529026") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:55:50.141541 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:50.141514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:50.142055 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:50.141567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:50.142055 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:50.141703    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle podName:0dbd943c-6d89-4b78-8b39-ba3bb18c0e53 nodeName:}" failed. No retries permitted until 2026-07-15 19:56:06.141683913 +0000 UTC m=+99.399086070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle") pod "router-default-79c74b977d-stx57" (UID: "0dbd943c-6d89-4b78-8b39-ba3bb18c0e53") : configmap references non-existent config key: service-ca.crt
Jul 15 19:55:50.144079 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:50.144053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-metrics-certs\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:55:56.288600 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:56.288557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:56.288600 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:56.288601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:56.289209 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:56.289190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f69ca9b-1411-4a49-9a5d-e533e3529026-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:56.291038 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:56.291022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f69ca9b-1411-4a49-9a5d-e533e3529026-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wttqf\" (UID: \"3f69ca9b-1411-4a49-9a5d-e533e3529026\") " pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:56.360771 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:56.360714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wttqf"
Jul 15 19:55:56.498331 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:56.498305    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wttqf"]
Jul 15 19:55:56.501493 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:55:56.501464    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f69ca9b_1411_4a49_9a5d_e533e3529026.slice/crio-5d1c826ca533bab8db62e5aac47581a45f8ea90e36919c06bf1fd4003f4faa54 WatchSource:0}: Error finding container 5d1c826ca533bab8db62e5aac47581a45f8ea90e36919c06bf1fd4003f4faa54: Status 404 returned error can't find the container with id 5d1c826ca533bab8db62e5aac47581a45f8ea90e36919c06bf1fd4003f4faa54
Jul 15 19:55:56.728594 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:56.728558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wttqf" event={"ID":"3f69ca9b-1411-4a49-9a5d-e533e3529026","Type":"ContainerStarted","Data":"5d1c826ca533bab8db62e5aac47581a45f8ea90e36919c06bf1fd4003f4faa54"}
Jul 15 19:55:59.292569 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:59.292535    2563 scope.go:117] "RemoveContainer" containerID="18d611bfe94b318636acfa83e2c9f18781a53dcd8ae2dd11b993d07117fc5b2a"
Jul 15 19:55:59.740982 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:59.740889    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 19:55:59.741417 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:59.741395    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/1.log"
Jul 15 19:55:59.741557 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:59.741440    2563 generic.go:356] "Generic (PLEG): container finished" podID="05d1c048-b150-4468-a377-1390dbe7a1ad" containerID="8af3a0102e88a2adf39fc256d8b505139c9feca507e477a8ab9b32972b2b3c2e" exitCode=255
Jul 15 19:55:59.741557 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:59.741496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pqfl4" event={"ID":"05d1c048-b150-4468-a377-1390dbe7a1ad","Type":"ContainerDied","Data":"8af3a0102e88a2adf39fc256d8b505139c9feca507e477a8ab9b32972b2b3c2e"}
Jul 15 19:55:59.741557 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:59.741552    2563 scope.go:117] "RemoveContainer" containerID="18d611bfe94b318636acfa83e2c9f18781a53dcd8ae2dd11b993d07117fc5b2a"
Jul 15 19:55:59.742027 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:55:59.742006    2563 scope.go:117] "RemoveContainer" containerID="8af3a0102e88a2adf39fc256d8b505139c9feca507e477a8ab9b32972b2b3c2e"
Jul 15 19:55:59.742218 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:55:59.742185    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-pqfl4_openshift-console-operator(05d1c048-b150-4468-a377-1390dbe7a1ad)\"" pod="openshift-console-operator/console-operator-7866497494-pqfl4" podUID="05d1c048-b150-4468-a377-1390dbe7a1ad"
Jul 15 19:56:02.752680 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:02.752644    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wttqf" event={"ID":"3f69ca9b-1411-4a49-9a5d-e533e3529026","Type":"ContainerStarted","Data":"e71a3463f144532f33820e3e64ae8916b10e0f32f7c97e058c02aaee1d007d93"}
Jul 15 19:56:02.754070 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:02.754050    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 19:56:04.406079 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.405588    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"]
Jul 15 19:56:04.417424 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.417398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.440782 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.440690    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 15 19:56:04.441021 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.441006    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 19:56:04.441114 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.441095    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 19:56:04.441156 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.441007    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 19:56:04.441318 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.441302    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 15 19:56:04.441358 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.441321    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 15 19:56:04.461707 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.461684    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 15 19:56:04.529830 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.529795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"]
Jul 15 19:56:04.557459 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.557088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.557459 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.557146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-hub\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.557459 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.557190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68c25d4-af68-47d5-87b5-38c487a7a0b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.557459 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.557225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.557459 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.557254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-ca\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.557459 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.557335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj6np\" (UniqueName: \"kubernetes.io/projected/d68c25d4-af68-47d5-87b5-38c487a7a0b2-kube-api-access-tj6np\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.657898 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.657812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj6np\" (UniqueName: \"kubernetes.io/projected/d68c25d4-af68-47d5-87b5-38c487a7a0b2-kube-api-access-tj6np\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.657898 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.657871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.658124 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.657902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-hub\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.658124 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.658032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68c25d4-af68-47d5-87b5-38c487a7a0b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.658124 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.658082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.658124 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.658103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-ca\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.659093 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.659061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68c25d4-af68-47d5-87b5-38c487a7a0b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.660912 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.660891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.661026 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.660990    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-hub\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.661087 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.661018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-ca\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.661162 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.661146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d68c25d4-af68-47d5-87b5-38c487a7a0b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.673902 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.673874    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:56:04.674041 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.673929    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:56:04.674264 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.674251    2563 scope.go:117] "RemoveContainer" containerID="8af3a0102e88a2adf39fc256d8b505139c9feca507e477a8ab9b32972b2b3c2e"
Jul 15 19:56:04.674486 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:56:04.674454    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-pqfl4_openshift-console-operator(05d1c048-b150-4468-a377-1390dbe7a1ad)\"" pod="openshift-console-operator/console-operator-7866497494-pqfl4" podUID="05d1c048-b150-4468-a377-1390dbe7a1ad"
Jul 15 19:56:04.680400 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.680374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj6np\" (UniqueName: \"kubernetes.io/projected/d68c25d4-af68-47d5-87b5-38c487a7a0b2-kube-api-access-tj6np\") pod \"cluster-proxy-proxy-agent-56f7c5b569-xk8jq\" (UID: \"d68c25d4-af68-47d5-87b5-38c487a7a0b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:04.740005 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:04.739966    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"
Jul 15 19:56:05.946786 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:05.946688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq"]
Jul 15 19:56:05.949392 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:56:05.949364    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd68c25d4_af68_47d5_87b5_38c487a7a0b2.slice/crio-4cd6b7a27dc0c6183d739d45d6b4356e0db739f8cce52e3e9532c1f82d024cd2 WatchSource:0}: Error finding container 4cd6b7a27dc0c6183d739d45d6b4356e0db739f8cce52e3e9532c1f82d024cd2: Status 404 returned error can't find the container with id 4cd6b7a27dc0c6183d739d45d6b4356e0db739f8cce52e3e9532c1f82d024cd2
Jul 15 19:56:06.070272 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.070166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:56:06.072764 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.072714    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd40f713-e53b-4ddd-806b-298eda2be912-metrics-tls\") pod \"dns-default-cwl2l\" (UID: \"dd40f713-e53b-4ddd-806b-298eda2be912\") " pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:56:06.079916 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.079892    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c5qnq\""
Jul 15 19:56:06.087189 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.087167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:56:06.171298 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.171258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:56:06.172069 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.172044    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dbd943c-6d89-4b78-8b39-ba3bb18c0e53-service-ca-bundle\") pod \"router-default-79c74b977d-stx57\" (UID: \"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53\") " pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:56:06.186275 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.186247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:56:06.265039 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.265017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwl2l"]
Jul 15 19:56:06.267444 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:56:06.267336    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd40f713_e53b_4ddd_806b_298eda2be912.slice/crio-6258393871e3cc08fc3a24477b31b7880203c5403f6ee0f3fc373770b29310f5 WatchSource:0}: Error finding container 6258393871e3cc08fc3a24477b31b7880203c5403f6ee0f3fc373770b29310f5: Status 404 returned error can't find the container with id 6258393871e3cc08fc3a24477b31b7880203c5403f6ee0f3fc373770b29310f5
Jul 15 19:56:06.340660 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.340576    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79c74b977d-stx57"]
Jul 15 19:56:06.343885 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:56:06.343860    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbd943c_6d89_4b78_8b39_ba3bb18c0e53.slice/crio-2634daf6d94b5baba0c4d52d87a1f4d3c0746df41fd9d27bfc08cf8a92ed0edd WatchSource:0}: Error finding container 2634daf6d94b5baba0c4d52d87a1f4d3c0746df41fd9d27bfc08cf8a92ed0edd: Status 404 returned error can't find the container with id 2634daf6d94b5baba0c4d52d87a1f4d3c0746df41fd9d27bfc08cf8a92ed0edd
Jul 15 19:56:06.771000 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.770963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79c74b977d-stx57" event={"ID":"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53","Type":"ContainerStarted","Data":"5eb979b1de518bf3beb03f33399d6d908e45f2877004cd8ff48ffb388a633632"}
Jul 15 19:56:06.771190 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.771020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79c74b977d-stx57" event={"ID":"0dbd943c-6d89-4b78-8b39-ba3bb18c0e53","Type":"ContainerStarted","Data":"2634daf6d94b5baba0c4d52d87a1f4d3c0746df41fd9d27bfc08cf8a92ed0edd"}
Jul 15 19:56:06.772302 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.772273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq" event={"ID":"d68c25d4-af68-47d5-87b5-38c487a7a0b2","Type":"ContainerStarted","Data":"4cd6b7a27dc0c6183d739d45d6b4356e0db739f8cce52e3e9532c1f82d024cd2"}
Jul 15 19:56:06.773524 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.773498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwl2l" event={"ID":"dd40f713-e53b-4ddd-806b-298eda2be912","Type":"ContainerStarted","Data":"6258393871e3cc08fc3a24477b31b7880203c5403f6ee0f3fc373770b29310f5"}
Jul 15 19:56:06.775203 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:06.775174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wttqf" event={"ID":"3f69ca9b-1411-4a49-9a5d-e533e3529026","Type":"ContainerStarted","Data":"ffb364aa02cd7ac110e4c2a9a8228e0b743653406eb52d5724d34a97cbe35aeb"}
Jul 15 19:56:07.186613 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:07.186570    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:56:07.190515 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:07.190475    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:56:07.231144 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:07.231089    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79c74b977d-stx57" podStartSLOduration=33.231073582 podStartE2EDuration="33.231073582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:06.839226561 +0000 UTC m=+100.096628740" watchObservedRunningTime="2026-07-15 19:56:07.231073582 +0000 UTC m=+100.488475760"
Jul 15 19:56:07.779536 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:07.779499    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:56:07.781143 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:07.781114    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79c74b977d-stx57"
Jul 15 19:56:09.665817 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:09.665787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hzrbl"
Jul 15 19:56:10.792424 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:10.792385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wttqf" event={"ID":"3f69ca9b-1411-4a49-9a5d-e533e3529026","Type":"ContainerStarted","Data":"b0c978c5bfaa832c6f4ee041446b943c2ffff2c40a70df237446f823390da37e"}
Jul 15 19:56:10.798241 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:10.798209    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwl2l" event={"ID":"dd40f713-e53b-4ddd-806b-298eda2be912","Type":"ContainerStarted","Data":"1784b26e4a06db8f79a06f105525513dc2ac14e8d18f063861ef0fb09d0c7a44"}
Jul 15 19:56:10.798375 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:10.798285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwl2l" event={"ID":"dd40f713-e53b-4ddd-806b-298eda2be912","Type":"ContainerStarted","Data":"66044a3ce2dedd5cc4b3ef662e7cbe8d1c3e7d7116271a2e7c55f9fb634c5e68"}
Jul 15 19:56:10.798375 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:10.798331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:56:10.878336 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:10.878288    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cwl2l" podStartSLOduration=65.17274147 podStartE2EDuration="1m8.878273469s" totalImagesPullingTime="3.705531999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:10.87788702 +0000 UTC m=+104.135289199" watchObservedRunningTime="2026-07-15 19:56:10.878273469 +0000 UTC m=+104.135675647"
Jul 15 19:56:10.878492 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:10.878372    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wttqf" podStartSLOduration=17.520432254 podStartE2EDuration="30.878366526s" totalImagesPullingTime="13.357934272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:55:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:10.82598615 +0000 UTC m=+104.083388328" watchObservedRunningTime="2026-07-15 19:56:10.878366526 +0000 UTC m=+104.135768706"
Jul 15 19:56:16.292015 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:16.291981    2563 scope.go:117] "RemoveContainer" containerID="8af3a0102e88a2adf39fc256d8b505139c9feca507e477a8ab9b32972b2b3c2e"
Jul 15 19:56:16.292561 ip-10-0-141-81 kubenswrapper[2563]: E0715 19:56:16.292175    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-pqfl4_openshift-console-operator(05d1c048-b150-4468-a377-1390dbe7a1ad)\"" pod="openshift-console-operator/console-operator-7866497494-pqfl4" podUID="05d1c048-b150-4468-a377-1390dbe7a1ad"
Jul 15 19:56:17.824252 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:17.824214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq" event={"ID":"d68c25d4-af68-47d5-87b5-38c487a7a0b2","Type":"ContainerStarted","Data":"cfcf4b8c05104f84571bbc406ace3a88c2d1cffd125ac4fbf0ac9b9cc2fb12ab"}
Jul 15 19:56:17.824638 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:17.824262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq" event={"ID":"d68c25d4-af68-47d5-87b5-38c487a7a0b2","Type":"ContainerStarted","Data":"8d1bdee96c36a7b4819d799071dcbeb01d394318b093e9bfe5908fb54dde09ff"}
Jul 15 19:56:17.824638 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:17.824275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq" event={"ID":"d68c25d4-af68-47d5-87b5-38c487a7a0b2","Type":"ContainerStarted","Data":"d33b82ff9b118c2469cc976506516a22a53769100f36edf7f7369909ed7c14f6"}
Jul 15 19:56:17.882185 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:17.882140    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f7c5b569-xk8jq" podStartSLOduration=2.904201921 podStartE2EDuration="13.882126711s" totalImagesPullingTime="10.97792479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:17.880042212 +0000 UTC m=+111.137444390" watchObservedRunningTime="2026-07-15 19:56:17.882126711 +0000 UTC m=+111.139528890"
Jul 15 19:56:19.048707 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.048680    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pwnpp"]
Jul 15 19:56:19.051055 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.051037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.061100 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.061076    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 19:56:19.061255 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.061238    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 19:56:19.064395 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.064382    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 19:56:19.064519 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.064507    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 19:56:19.064635 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.064621    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 19:56:19.071104 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.071086    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 19:56:19.077913 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.077888    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dpkjd\""
Jul 15 19:56:19.175812 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.175785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-sys\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.175936 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.175816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.175936 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.175837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-root\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.176031 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.175941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.176031 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.175991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-wtmp\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.176031 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.176015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79xrj\" (UniqueName: \"kubernetes.io/projected/68f7322c-57e7-4676-adeb-4d2074c2a6c9-kube-api-access-79xrj\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.176149 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.176046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-textfile\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.176149 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.176061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-tls\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.176149 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.176086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68f7322c-57e7-4676-adeb-4d2074c2a6c9-metrics-client-ca\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276625 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-root\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276785 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276785 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-root\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276785 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-wtmp\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276785 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79xrj\" (UniqueName: \"kubernetes.io/projected/68f7322c-57e7-4676-adeb-4d2074c2a6c9-kube-api-access-79xrj\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-textfile\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-tls\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68f7322c-57e7-4676-adeb-4d2074c2a6c9-metrics-client-ca\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-sys\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-wtmp\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.276987 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.276969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68f7322c-57e7-4676-adeb-4d2074c2a6c9-sys\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.277338 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.277310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-textfile\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.277620 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.277502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.277620 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.277502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68f7322c-57e7-4676-adeb-4d2074c2a6c9-metrics-client-ca\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.279093 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.279073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.279179 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.279163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68f7322c-57e7-4676-adeb-4d2074c2a6c9-node-exporter-tls\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.315905 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.315847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79xrj\" (UniqueName: \"kubernetes.io/projected/68f7322c-57e7-4676-adeb-4d2074c2a6c9-kube-api-access-79xrj\") pod \"node-exporter-pwnpp\" (UID: \"68f7322c-57e7-4676-adeb-4d2074c2a6c9\") " pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.359846 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.359817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwnpp"
Jul 15 19:56:19.369548 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:56:19.369523    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f7322c_57e7_4676_adeb_4d2074c2a6c9.slice/crio-b55115d46d08ad5d86077d713ef43b41a23510e749305a8b5af0d62f8b0e6f0e WatchSource:0}: Error finding container b55115d46d08ad5d86077d713ef43b41a23510e749305a8b5af0d62f8b0e6f0e: Status 404 returned error can't find the container with id b55115d46d08ad5d86077d713ef43b41a23510e749305a8b5af0d62f8b0e6f0e
Jul 15 19:56:19.833599 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:19.833558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwnpp" event={"ID":"68f7322c-57e7-4676-adeb-4d2074c2a6c9","Type":"ContainerStarted","Data":"b55115d46d08ad5d86077d713ef43b41a23510e749305a8b5af0d62f8b0e6f0e"}
Jul 15 19:56:20.803507 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:20.803434    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cwl2l"
Jul 15 19:56:20.838554 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:20.838525    2563 generic.go:356] "Generic (PLEG): container finished" podID="68f7322c-57e7-4676-adeb-4d2074c2a6c9" containerID="3b54b46d5052ce9c52aaa27380f542674a28c8fd29ce1bc8678a991e28c20133" exitCode=0
Jul 15 19:56:20.838695 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:20.838580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwnpp" event={"ID":"68f7322c-57e7-4676-adeb-4d2074c2a6c9","Type":"ContainerDied","Data":"3b54b46d5052ce9c52aaa27380f542674a28c8fd29ce1bc8678a991e28c20133"}
Jul 15 19:56:21.843902 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:21.843869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwnpp" event={"ID":"68f7322c-57e7-4676-adeb-4d2074c2a6c9","Type":"ContainerStarted","Data":"33adf4c88076e4a982c1d5785cc41b84811fd89999ffab42596de3c0f20d34f3"}
Jul 15 19:56:21.844274 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:21.843921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwnpp" event={"ID":"68f7322c-57e7-4676-adeb-4d2074c2a6c9","Type":"ContainerStarted","Data":"102ed5e04fe55efdfb23164a73e4848db6bea09e6ab41f75eedf6066954d3b53"}
Jul 15 19:56:21.883885 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:21.883832    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pwnpp" podStartSLOduration=2.819851881 podStartE2EDuration="3.88381526s" totalImagesPullingTime="991.044587ms" totalInitContainerRuntime="72.918792ms" isStatefulPod=true podCreationTimestamp="2026-07-15 19:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:21.883235685 +0000 UTC m=+115.140637875" watchObservedRunningTime="2026-07-15 19:56:21.88381526 +0000 UTC m=+115.141217439"
Jul 15 19:56:28.292439 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:28.292406    2563 scope.go:117] "RemoveContainer" containerID="8af3a0102e88a2adf39fc256d8b505139c9feca507e477a8ab9b32972b2b3c2e"
Jul 15 19:56:28.868644 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:28.868614    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 19:56:28.868835 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:28.868699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-pqfl4" event={"ID":"05d1c048-b150-4468-a377-1390dbe7a1ad","Type":"ContainerStarted","Data":"ec5b241543635b7d3034cab27f0864646accb65507f4acda6253e6b8bee0f30c"}
Jul 15 19:56:28.869051 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:28.869031    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:56:28.873921 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:28.873893    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-pqfl4"
Jul 15 19:56:28.909487 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:28.909419    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-pqfl4" podStartSLOduration=49.812183338 podStartE2EDuration="54.909396444s" totalImagesPullingTime="5.097213106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:28.908425894 +0000 UTC m=+122.165828073" watchObservedRunningTime="2026-07-15 19:56:28.909396444 +0000 UTC m=+122.166798623"
Jul 15 19:56:37.116862 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:37.116823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:56:37.119319 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:37.119299    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760b10f9-2f25-4197-9678-518c2f396178-metrics-certs\") pod \"network-metrics-daemon-kc8fr\" (UID: \"760b10f9-2f25-4197-9678-518c2f396178\") " pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:56:37.417656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:37.417578    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjg99\""
Jul 15 19:56:37.425514 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:37.425496    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kc8fr"
Jul 15 19:56:37.582074 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:37.582046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kc8fr"]
Jul 15 19:56:37.584359 ip-10-0-141-81 kubenswrapper[2563]: W0715 19:56:37.584327    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760b10f9_2f25_4197_9678_518c2f396178.slice/crio-e399838540bf199a6eab834041f6f3f26e5985196854dbb16114a1188ce567f2 WatchSource:0}: Error finding container e399838540bf199a6eab834041f6f3f26e5985196854dbb16114a1188ce567f2: Status 404 returned error can't find the container with id e399838540bf199a6eab834041f6f3f26e5985196854dbb16114a1188ce567f2
Jul 15 19:56:37.898656 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:37.898619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kc8fr" event={"ID":"760b10f9-2f25-4197-9678-518c2f396178","Type":"ContainerStarted","Data":"e399838540bf199a6eab834041f6f3f26e5985196854dbb16114a1188ce567f2"}
Jul 15 19:56:38.904197 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:38.904164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kc8fr" event={"ID":"760b10f9-2f25-4197-9678-518c2f396178","Type":"ContainerStarted","Data":"6dce491635499216e57df6809d1b434bcb4bd2c293d278a7e934f24671d37fd3"}
Jul 15 19:56:38.904652 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:38.904217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kc8fr" event={"ID":"760b10f9-2f25-4197-9678-518c2f396178","Type":"ContainerStarted","Data":"17f3cdd6a96141294140a613fd42ada2df8f8be34fe61293e25d41775aa2625e"}
Jul 15 19:56:38.935244 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:56:38.935199    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kc8fr" podStartSLOduration=131.093887567 podStartE2EDuration="2m11.935181678s" totalImagesPullingTime="841.294111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:38.934156051 +0000 UTC m=+132.191558223" watchObservedRunningTime="2026-07-15 19:56:38.935181678 +0000 UTC m=+132.192583895"
Jul 15 19:59:27.114325 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:59:27.114295    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 19:59:27.116569 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:59:27.116543    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 19:59:27.121497 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:59:27.121474    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 19:59:27.124245 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:59:27.124225    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 19:59:27.124927 ip-10-0-141-81 kubenswrapper[2563]: I0715 19:59:27.124908    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 20:00:25.118562 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.118484    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vqbxg"]
Jul 15 20:00:25.121360 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.121344    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.124395 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.124372    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 20:00:25.124958 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.124937    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 20:00:25.125554 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.125540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9ttkb\""
Jul 15 20:00:25.134367 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.134347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vqbxg"]
Jul 15 20:00:25.192376 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.192338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x77nb\" (UniqueName: \"kubernetes.io/projected/977f02e4-756c-4d15-9c4e-0d20f7e2a677-kube-api-access-x77nb\") pod \"cert-manager-webhook-6486ffd997-vqbxg\" (UID: \"977f02e4-756c-4d15-9c4e-0d20f7e2a677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.192553 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.192385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/977f02e4-756c-4d15-9c4e-0d20f7e2a677-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vqbxg\" (UID: \"977f02e4-756c-4d15-9c4e-0d20f7e2a677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.292777 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.292747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x77nb\" (UniqueName: \"kubernetes.io/projected/977f02e4-756c-4d15-9c4e-0d20f7e2a677-kube-api-access-x77nb\") pod \"cert-manager-webhook-6486ffd997-vqbxg\" (UID: \"977f02e4-756c-4d15-9c4e-0d20f7e2a677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.292952 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.292793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/977f02e4-756c-4d15-9c4e-0d20f7e2a677-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vqbxg\" (UID: \"977f02e4-756c-4d15-9c4e-0d20f7e2a677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.303213 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.303187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/977f02e4-756c-4d15-9c4e-0d20f7e2a677-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vqbxg\" (UID: \"977f02e4-756c-4d15-9c4e-0d20f7e2a677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.303663 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.303645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x77nb\" (UniqueName: \"kubernetes.io/projected/977f02e4-756c-4d15-9c4e-0d20f7e2a677-kube-api-access-x77nb\") pod \"cert-manager-webhook-6486ffd997-vqbxg\" (UID: \"977f02e4-756c-4d15-9c4e-0d20f7e2a677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.430789 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.430691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:25.559210 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.559180    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vqbxg"]
Jul 15 20:00:25.561394 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:00:25.561368    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod977f02e4_756c_4d15_9c4e_0d20f7e2a677.slice/crio-57dc2c014fa07b67860cc7e98a2b982a5f6ea66518b656f28e0189c4129e8a23 WatchSource:0}: Error finding container 57dc2c014fa07b67860cc7e98a2b982a5f6ea66518b656f28e0189c4129e8a23: Status 404 returned error can't find the container with id 57dc2c014fa07b67860cc7e98a2b982a5f6ea66518b656f28e0189c4129e8a23
Jul 15 20:00:25.563366 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.563351    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:00:25.631941 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:25.631908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg" event={"ID":"977f02e4-756c-4d15-9c4e-0d20f7e2a677","Type":"ContainerStarted","Data":"57dc2c014fa07b67860cc7e98a2b982a5f6ea66518b656f28e0189c4129e8a23"}
Jul 15 20:00:28.644628 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:28.644593    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg" event={"ID":"977f02e4-756c-4d15-9c4e-0d20f7e2a677","Type":"ContainerStarted","Data":"b2729b047592657eb985688c9dfdb551684e3b7c2cb8bcc363786969c91fd7ce"}
Jul 15 20:00:28.645038 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:28.644742    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:00:28.676228 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:28.676128    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg" podStartSLOduration=0.8257862 podStartE2EDuration="3.676113106s" totalImagesPullingTime="2.850326906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:28.67570688 +0000 UTC m=+361.933109058" watchObservedRunningTime="2026-07-15 20:00:28.676113106 +0000 UTC m=+361.933515284"
Jul 15 20:00:34.651198 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:00:34.651170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vqbxg"
Jul 15 20:01:05.839640 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.839604    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"]
Jul 15 20:01:05.842929 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.842905    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:05.848631 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.848608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 20:01:05.848888 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.848865    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 20:01:05.848996 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.848934    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 20:01:05.849877 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.849565    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kg598\""
Jul 15 20:01:05.849877 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.849583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 20:01:05.863627 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.863597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"]
Jul 15 20:01:05.989293 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.989255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12d9f750-efd2-446c-bc99-fb5a422a683e-tmp\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:05.989475 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.989314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12d9f750-efd2-446c-bc99-fb5a422a683e-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:05.989475 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.989391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12d9f750-efd2-446c-bc99-fb5a422a683e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:05.989475 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.989458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12d9f750-efd2-446c-bc99-fb5a422a683e-manifests\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:05.989575 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:05.989501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rf77\" (UniqueName: \"kubernetes.io/projected/12d9f750-efd2-446c-bc99-fb5a422a683e-kube-api-access-4rf77\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.090584 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.090491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12d9f750-efd2-446c-bc99-fb5a422a683e-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.090584 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.090524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12d9f750-efd2-446c-bc99-fb5a422a683e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.090584 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.090548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12d9f750-efd2-446c-bc99-fb5a422a683e-manifests\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.090899 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.090672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rf77\" (UniqueName: \"kubernetes.io/projected/12d9f750-efd2-446c-bc99-fb5a422a683e-kube-api-access-4rf77\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.090899 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.090741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12d9f750-efd2-446c-bc99-fb5a422a683e-tmp\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.090899 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.090895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12d9f750-efd2-446c-bc99-fb5a422a683e-manifests\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.093074 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.093042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12d9f750-efd2-446c-bc99-fb5a422a683e-tmp\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.093206 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.093155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12d9f750-efd2-446c-bc99-fb5a422a683e-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.093306 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.093282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12d9f750-efd2-446c-bc99-fb5a422a683e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.100232 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.100208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rf77\" (UniqueName: \"kubernetes.io/projected/12d9f750-efd2-446c-bc99-fb5a422a683e-kube-api-access-4rf77\") pod \"opendatahub-operator-controller-manager-57c4dc5856-x9bx2\" (UID: \"12d9f750-efd2-446c-bc99-fb5a422a683e\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.153266 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.153230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:06.297473 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.297437    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"]
Jul 15 20:01:06.300854 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:01:06.300823    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d9f750_efd2_446c_bc99_fb5a422a683e.slice/crio-7f15e7b5130175a6f2eb5a4e3f695aa075e64bbd2c6362ffdb80d6053ee53869 WatchSource:0}: Error finding container 7f15e7b5130175a6f2eb5a4e3f695aa075e64bbd2c6362ffdb80d6053ee53869: Status 404 returned error can't find the container with id 7f15e7b5130175a6f2eb5a4e3f695aa075e64bbd2c6362ffdb80d6053ee53869
Jul 15 20:01:06.770586 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:06.770498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2" event={"ID":"12d9f750-efd2-446c-bc99-fb5a422a683e","Type":"ContainerStarted","Data":"7f15e7b5130175a6f2eb5a4e3f695aa075e64bbd2c6362ffdb80d6053ee53869"}
Jul 15 20:01:10.786853 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:10.786814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2" event={"ID":"12d9f750-efd2-446c-bc99-fb5a422a683e","Type":"ContainerStarted","Data":"e6da7802ea79d6b86ffcc15ce365cdb237cded2d388a61b350b535b7e07ffd4b"}
Jul 15 20:01:11.791324 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:11.791283    2563 generic.go:356] "Generic (PLEG): container finished" podID="12d9f750-efd2-446c-bc99-fb5a422a683e" containerID="e6da7802ea79d6b86ffcc15ce365cdb237cded2d388a61b350b535b7e07ffd4b" exitCode=0
Jul 15 20:01:11.791808 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:11.791335    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2" event={"ID":"12d9f750-efd2-446c-bc99-fb5a422a683e","Type":"ContainerDied","Data":"e6da7802ea79d6b86ffcc15ce365cdb237cded2d388a61b350b535b7e07ffd4b"}
Jul 15 20:01:14.809784 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:14.809741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2" event={"ID":"12d9f750-efd2-446c-bc99-fb5a422a683e","Type":"ContainerStarted","Data":"574c28eca18b6d84368b4635aa8c6e69723222e08bb051c92af6e08ed2d980e2"}
Jul 15 20:01:14.810194 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:14.809848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:14.837206 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:14.837145    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2" podStartSLOduration=2.92711625 podStartE2EDuration="9.837127485s" totalImagesPullingTime="5.827023523s" totalInitContainerRuntime="1.082987712s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:14.836639961 +0000 UTC m=+408.094042155" watchObservedRunningTime="2026-07-15 20:01:14.837127485 +0000 UTC m=+408.094529664"
Jul 15 20:01:25.816656 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:25.816626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-x9bx2"
Jul 15 20:01:28.095590 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.095559    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"]
Jul 15 20:01:28.101726 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.101694    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.106174 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.106153    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 20:01:28.107881 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.107852    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 20:01:28.108002 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.107988    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xlz4z\""
Jul 15 20:01:28.115402 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.115361    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"]
Jul 15 20:01:28.267749 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.267679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z76f6\" (UniqueName: \"kubernetes.io/projected/e6e9553d-adc7-4826-9841-c2a03eb4f771-kube-api-access-z76f6\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.267942 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.267768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e9553d-adc7-4826-9841-c2a03eb4f771-tmp\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.267942 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.267808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9553d-adc7-4826-9841-c2a03eb4f771-tls-certs\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.317778 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.317742    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpzb7"]
Jul 15 20:01:28.319750 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.319709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:28.322620 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.322599    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vjgb6\""
Jul 15 20:01:28.323203 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.323190    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 15 20:01:28.334363 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.334338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpzb7"]
Jul 15 20:01:28.368219 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.368185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eab0a56-a6cb-43b9-a113-6a566ef1d929-cert\") pod \"odh-model-controller-7998c67dc4-xpzb7\" (UID: \"9eab0a56-a6cb-43b9-a113-6a566ef1d929\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:28.368422 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.368225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc7x4\" (UniqueName: \"kubernetes.io/projected/9eab0a56-a6cb-43b9-a113-6a566ef1d929-kube-api-access-qc7x4\") pod \"odh-model-controller-7998c67dc4-xpzb7\" (UID: \"9eab0a56-a6cb-43b9-a113-6a566ef1d929\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:28.368422 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.368262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z76f6\" (UniqueName: \"kubernetes.io/projected/e6e9553d-adc7-4826-9841-c2a03eb4f771-kube-api-access-z76f6\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.368422 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.368299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e9553d-adc7-4826-9841-c2a03eb4f771-tmp\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.368422 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.368343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9553d-adc7-4826-9841-c2a03eb4f771-tls-certs\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.370901 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.370876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9553d-adc7-4826-9841-c2a03eb4f771-tls-certs\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.370901 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.370888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e9553d-adc7-4826-9841-c2a03eb4f771-tmp\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.385782 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.385706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z76f6\" (UniqueName: \"kubernetes.io/projected/e6e9553d-adc7-4826-9841-c2a03eb4f771-kube-api-access-z76f6\") pod \"kube-auth-proxy-7c4bdf9b6b-jvzrq\" (UID: \"e6e9553d-adc7-4826-9841-c2a03eb4f771\") " pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.416706 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.416661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"
Jul 15 20:01:28.469248 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.469032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eab0a56-a6cb-43b9-a113-6a566ef1d929-cert\") pod \"odh-model-controller-7998c67dc4-xpzb7\" (UID: \"9eab0a56-a6cb-43b9-a113-6a566ef1d929\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:28.469248 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.469086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc7x4\" (UniqueName: \"kubernetes.io/projected/9eab0a56-a6cb-43b9-a113-6a566ef1d929-kube-api-access-qc7x4\") pod \"odh-model-controller-7998c67dc4-xpzb7\" (UID: \"9eab0a56-a6cb-43b9-a113-6a566ef1d929\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:28.469441 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:01:28.469285    2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 20:01:28.469441 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:01:28.469347    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eab0a56-a6cb-43b9-a113-6a566ef1d929-cert podName:9eab0a56-a6cb-43b9-a113-6a566ef1d929 nodeName:}" failed. No retries permitted until 2026-07-15 20:01:28.969326278 +0000 UTC m=+422.226728435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9eab0a56-a6cb-43b9-a113-6a566ef1d929-cert") pod "odh-model-controller-7998c67dc4-xpzb7" (UID: "9eab0a56-a6cb-43b9-a113-6a566ef1d929") : secret "odh-model-controller-webhook-cert" not found
Jul 15 20:01:28.487190 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.487127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc7x4\" (UniqueName: \"kubernetes.io/projected/9eab0a56-a6cb-43b9-a113-6a566ef1d929-kube-api-access-qc7x4\") pod \"odh-model-controller-7998c67dc4-xpzb7\" (UID: \"9eab0a56-a6cb-43b9-a113-6a566ef1d929\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:28.573578 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.573542    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq"]
Jul 15 20:01:28.576887 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:01:28.576855    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e9553d_adc7_4826_9841_c2a03eb4f771.slice/crio-5a203e33daff1b6d184804426b6268817a4221700a1827e7e663f36e2f78147c WatchSource:0}: Error finding container 5a203e33daff1b6d184804426b6268817a4221700a1827e7e663f36e2f78147c: Status 404 returned error can't find the container with id 5a203e33daff1b6d184804426b6268817a4221700a1827e7e663f36e2f78147c
Jul 15 20:01:28.861530 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.861482    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq" event={"ID":"e6e9553d-adc7-4826-9841-c2a03eb4f771","Type":"ContainerStarted","Data":"5a203e33daff1b6d184804426b6268817a4221700a1827e7e663f36e2f78147c"}
Jul 15 20:01:28.972966 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.972928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eab0a56-a6cb-43b9-a113-6a566ef1d929-cert\") pod \"odh-model-controller-7998c67dc4-xpzb7\" (UID: \"9eab0a56-a6cb-43b9-a113-6a566ef1d929\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:28.975579 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:28.975549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eab0a56-a6cb-43b9-a113-6a566ef1d929-cert\") pod \"odh-model-controller-7998c67dc4-xpzb7\" (UID: \"9eab0a56-a6cb-43b9-a113-6a566ef1d929\") " pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:29.230222 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:29.230130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:29.388615 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:29.388572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xpzb7"]
Jul 15 20:01:29.392598 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:01:29.392557    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eab0a56_a6cb_43b9_a113_6a566ef1d929.slice/crio-0aeb6397421ba5721a26c859ff54a9ad994808629f070f03c883e72336550c16 WatchSource:0}: Error finding container 0aeb6397421ba5721a26c859ff54a9ad994808629f070f03c883e72336550c16: Status 404 returned error can't find the container with id 0aeb6397421ba5721a26c859ff54a9ad994808629f070f03c883e72336550c16
Jul 15 20:01:29.866957 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:29.866921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7" event={"ID":"9eab0a56-a6cb-43b9-a113-6a566ef1d929","Type":"ContainerStarted","Data":"0aeb6397421ba5721a26c859ff54a9ad994808629f070f03c883e72336550c16"}
Jul 15 20:01:29.997906 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:29.997862    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"]
Jul 15 20:01:30.000096 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.000081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.005919 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.005894    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 20:01:30.006088 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.005958    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 20:01:30.006088 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.005958    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 20:01:30.006088 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.006020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 20:01:30.006088 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.006042    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dcj9h\""
Jul 15 20:01:30.018319 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.018292    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"]
Jul 15 20:01:30.083925 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.083881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.083925 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.083946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kt4j\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-kube-api-access-9kt4j\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.084210 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.084001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-local-certs\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.084210 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.084027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-cacerts\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.084210 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.084108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-token\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.084210 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.084143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.084210 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.084169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.185469 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.185377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kt4j\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-kube-api-access-9kt4j\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.185469 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.185450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-local-certs\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.185701 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.185486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-cacerts\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.185701 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.185525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-token\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.185701 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.185564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.185701 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.185589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.185701 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.185657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.186442 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.186407    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.189315 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.189287    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.189879 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.189823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-cacerts\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.191120 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.191099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-local-certs\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.192360 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.192337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.195596 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.195551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kt4j\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-kube-api-access-9kt4j\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.199339 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.199316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-token\") pod \"istiod-openshift-gateway-7574459656-nmwk9\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.309708 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.309674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:30.539220 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.539079    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"]
Jul 15 20:01:30.546467 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:01:30.546433    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde7ec682_27e0_41bc_acc0_ec1e8dfbc5f4.slice/crio-71e5e33c2e75c1653408f831bae430e3379875edb5ddafb2c3807e39f16f8c9a WatchSource:0}: Error finding container 71e5e33c2e75c1653408f831bae430e3379875edb5ddafb2c3807e39f16f8c9a: Status 404 returned error can't find the container with id 71e5e33c2e75c1653408f831bae430e3379875edb5ddafb2c3807e39f16f8c9a
Jul 15 20:01:30.872789 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:30.872752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9" event={"ID":"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4","Type":"ContainerStarted","Data":"71e5e33c2e75c1653408f831bae430e3379875edb5ddafb2c3807e39f16f8c9a"}
Jul 15 20:01:33.888035 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:33.887989    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7" event={"ID":"9eab0a56-a6cb-43b9-a113-6a566ef1d929","Type":"ContainerStarted","Data":"b8b83f7d5317284cf0e6367d0b7141916771012bb6ccf459e030b6cb77b865e0"}
Jul 15 20:01:33.888529 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:33.888064    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:33.889459 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:33.889433    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq" event={"ID":"e6e9553d-adc7-4826-9841-c2a03eb4f771","Type":"ContainerStarted","Data":"f11ccaf0529e362528b8495ea3b44419a49da748640bd622302cd9fe662a5bac"}
Jul 15 20:01:33.912052 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:33.911999    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7" podStartSLOduration=2.308863785 podStartE2EDuration="5.911982388s" totalImagesPullingTime="3.603118603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:33.910846457 +0000 UTC m=+427.168248637" watchObservedRunningTime="2026-07-15 20:01:33.911982388 +0000 UTC m=+427.169384566"
Jul 15 20:01:33.940971 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:33.940915    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c4bdf9b6b-jvzrq" podStartSLOduration=1.522739161 podStartE2EDuration="5.940899301s" totalImagesPullingTime="4.41816014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:33.940370234 +0000 UTC m=+427.197772413" watchObservedRunningTime="2026-07-15 20:01:33.940899301 +0000 UTC m=+427.198301479"
Jul 15 20:01:35.786271 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:35.786233    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 15 20:01:35.786516 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:35.786306    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 15 20:01:35.902230 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:35.902197    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9" event={"ID":"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4","Type":"ContainerStarted","Data":"5d22ef0d9fdf2a15074509e17a7a37650278076417ea969db24fc54f5b1c4f94"}
Jul 15 20:01:35.902423 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:35.902347    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:35.940412 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:35.940339    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9" podStartSLOduration=1.703492302 podStartE2EDuration="6.940321343s" totalImagesPullingTime="5.236829041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:35.938431515 +0000 UTC m=+429.195833693" watchObservedRunningTime="2026-07-15 20:01:35.940321343 +0000 UTC m=+429.197723521"
Jul 15 20:01:36.495867 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.495833    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"]
Jul 15 20:01:36.499116 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.499098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.502517 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.502493    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 20:01:36.503508 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.503480    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 20:01:36.503765 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.503746    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 20:01:36.503860 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.503748    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 20:01:36.503915 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.503861    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 20:01:36.504933 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.504739    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-z48ch\""
Jul 15 20:01:36.511825 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.511801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"]
Jul 15 20:01:36.541289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.541258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l49xd\" (UniqueName: \"kubernetes.io/projected/12adb0c6-daac-4705-9e2c-c2084f55000c-kube-api-access-l49xd\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.541471 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.541305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12adb0c6-daac-4705-9e2c-c2084f55000c-cert\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.541471 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.541328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12adb0c6-daac-4705-9e2c-c2084f55000c-metrics-cert\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.541471 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.541399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12adb0c6-daac-4705-9e2c-c2084f55000c-manager-config\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.642173 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.642128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12adb0c6-daac-4705-9e2c-c2084f55000c-manager-config\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.642359 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.642191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l49xd\" (UniqueName: \"kubernetes.io/projected/12adb0c6-daac-4705-9e2c-c2084f55000c-kube-api-access-l49xd\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.642359 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.642227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12adb0c6-daac-4705-9e2c-c2084f55000c-cert\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.642359 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.642247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12adb0c6-daac-4705-9e2c-c2084f55000c-metrics-cert\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.642848 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.642827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12adb0c6-daac-4705-9e2c-c2084f55000c-manager-config\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.645046 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.645014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12adb0c6-daac-4705-9e2c-c2084f55000c-metrics-cert\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.645046 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.645020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12adb0c6-daac-4705-9e2c-c2084f55000c-cert\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.657229 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.657202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l49xd\" (UniqueName: \"kubernetes.io/projected/12adb0c6-daac-4705-9e2c-c2084f55000c-kube-api-access-l49xd\") pod \"lws-controller-manager-7b4664dc78-77d8b\" (UID: \"12adb0c6-daac-4705-9e2c-c2084f55000c\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.809476 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.809369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:36.913228 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.913194    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-nmwk9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 20:01:36.913381 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.913251    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9" podUID="de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 20:01:36.986913 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:36.986777    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"]
Jul 15 20:01:36.994812 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:01:36.994773    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12adb0c6_daac_4705_9e2c_c2084f55000c.slice/crio-de1c8d86250182e72f647a2601a367188b35766cd76278f7dc01a9f2a2bdc620 WatchSource:0}: Error finding container de1c8d86250182e72f647a2601a367188b35766cd76278f7dc01a9f2a2bdc620: Status 404 returned error can't find the container with id de1c8d86250182e72f647a2601a367188b35766cd76278f7dc01a9f2a2bdc620
Jul 15 20:01:37.916855 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:37.916818    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b" event={"ID":"12adb0c6-daac-4705-9e2c-c2084f55000c","Type":"ContainerStarted","Data":"de1c8d86250182e72f647a2601a367188b35766cd76278f7dc01a9f2a2bdc620"}
Jul 15 20:01:39.913295 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:39.913263    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:39.926912 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:39.926876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b" event={"ID":"12adb0c6-daac-4705-9e2c-c2084f55000c","Type":"ContainerStarted","Data":"c6eb37c07869acf572acad7e11eeac674fc811fae71b975c42b5cf2ec7251781"}
Jul 15 20:01:39.927104 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:39.926935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:01:39.975297 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:39.975231    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b" podStartSLOduration=1.838715241 podStartE2EDuration="3.975209094s" totalImagesPullingTime="2.136493853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:39.97340259 +0000 UTC m=+433.230804772" watchObservedRunningTime="2026-07-15 20:01:39.975209094 +0000 UTC m=+433.232611274"
Jul 15 20:01:40.801929 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:40.801888    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"]
Jul 15 20:01:40.802159 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:40.802108    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9" podUID="de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" containerName="discovery" containerID="cri-o://5d22ef0d9fdf2a15074509e17a7a37650278076417ea969db24fc54f5b1c4f94" gracePeriod=30
Jul 15 20:01:40.932568 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:40.932536    2563 generic.go:356] "Generic (PLEG): container finished" podID="de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" containerID="5d22ef0d9fdf2a15074509e17a7a37650278076417ea969db24fc54f5b1c4f94" exitCode=0
Jul 15 20:01:40.932970 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:40.932612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9" event={"ID":"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4","Type":"ContainerDied","Data":"5d22ef0d9fdf2a15074509e17a7a37650278076417ea969db24fc54f5b1c4f94"}
Jul 15 20:01:41.046095 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.046071    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:41.081649 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.081566    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-token\") pod \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") "
Jul 15 20:01:41.081649 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.081610    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-local-certs\") pod \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") "
Jul 15 20:01:41.081898 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.081652    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-cacerts\") pod \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") "
Jul 15 20:01:41.081898 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.081699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-ca-configmap\") pod \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") "
Jul 15 20:01:41.081898 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.081751    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-kubeconfig\") pod \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") "
Jul 15 20:01:41.081898 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.081788    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kt4j\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-kube-api-access-9kt4j\") pod \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") "
Jul 15 20:01:41.081898 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.081813    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-dns-cert\") pod \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\" (UID: \"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4\") "
Jul 15 20:01:41.082142 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.082111    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" (UID: "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:41.085050 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.084976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" (UID: "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:41.085050 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.085027    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" (UID: "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:41.085311 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.085290    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-local-certs" (OuterVolumeSpecName: "local-certs") pod "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" (UID: "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:01:41.085369 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.085317    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-kube-api-access-9kt4j" (OuterVolumeSpecName: "kube-api-access-9kt4j") pod "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" (UID: "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4"). InnerVolumeSpecName "kube-api-access-9kt4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:01:41.085422 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.085412    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-cacerts" (OuterVolumeSpecName: "cacerts") pod "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" (UID: "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:41.085485 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.085468    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" (UID: "de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:01:41.182873 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.182830    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kt4j\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-kube-api-access-9kt4j\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:01:41.182873 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.182865    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-dns-cert\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:01:41.182873 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.182878    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-token\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:01:41.183125 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.182889    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-local-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:01:41.183125 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.182900    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-cacerts\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:01:41.183125 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.182911    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-csr-ca-configmap\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:01:41.183125 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.182922    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4-istio-kubeconfig\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:01:41.937189 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.937162    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"
Jul 15 20:01:41.937189 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.937182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9" event={"ID":"de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4","Type":"ContainerDied","Data":"71e5e33c2e75c1653408f831bae430e3379875edb5ddafb2c3807e39f16f8c9a"}
Jul 15 20:01:41.937746 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.937243    2563 scope.go:117] "RemoveContainer" containerID="5d22ef0d9fdf2a15074509e17a7a37650278076417ea969db24fc54f5b1c4f94"
Jul 15 20:01:41.960304 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.960274    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"]
Jul 15 20:01:41.963230 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:41.963206    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nmwk9"]
Jul 15 20:01:43.296771 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:43.296738    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" path="/var/lib/kubelet/pods/de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4/volumes"
Jul 15 20:01:44.937203 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:44.937170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xpzb7"
Jul 15 20:01:50.935416 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:01:50.935335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-77d8b"
Jul 15 20:03:08.650320 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.650284    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"]
Jul 15 20:03:08.650775 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.650552    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" containerName="discovery"
Jul 15 20:03:08.650775 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.650562    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" containerName="discovery"
Jul 15 20:03:08.650775 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.650605    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de7ec682-27e0-41bc-acc0-ec1e8dfbc5f4" containerName="discovery"
Jul 15 20:03:08.652627 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.652612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.655058 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.655036    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 15 20:03:08.655189 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.655041    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 15 20:03:08.655498 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.655473    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 20:03:08.655926 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.655907    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zhckq\""
Jul 15 20:03:08.656022 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.655907    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 20:03:08.665622 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.665600    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"]
Jul 15 20:03:08.778281 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.778251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxv2d\" (UniqueName: \"kubernetes.io/projected/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-kube-api-access-vxv2d\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.778481 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.778318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.778481 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.778416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.879229 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.879196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.879375 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.879249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.879375 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.879276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxv2d\" (UniqueName: \"kubernetes.io/projected/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-kube-api-access-vxv2d\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.879821 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.879801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.881752 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.881734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.893109 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.893083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxv2d\" (UniqueName: \"kubernetes.io/projected/b4705a9b-7f00-47a1-8144-d88e3b97f8ed-kube-api-access-vxv2d\") pod \"kuadrant-console-plugin-66b5774985-8h7tn\" (UID: \"b4705a9b-7f00-47a1-8144-d88e3b97f8ed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:08.962172 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:08.962092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"
Jul 15 20:03:09.097422 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:09.097394    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn"]
Jul 15 20:03:09.100634 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:03:09.100601    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4705a9b_7f00_47a1_8144_d88e3b97f8ed.slice/crio-bda0d8bb0311e82bb43d40db54070ce8e75c1d2e49bce76fd3b99040e3ff8b90 WatchSource:0}: Error finding container bda0d8bb0311e82bb43d40db54070ce8e75c1d2e49bce76fd3b99040e3ff8b90: Status 404 returned error can't find the container with id bda0d8bb0311e82bb43d40db54070ce8e75c1d2e49bce76fd3b99040e3ff8b90
Jul 15 20:03:09.251543 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:09.251458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn" event={"ID":"b4705a9b-7f00-47a1-8144-d88e3b97f8ed","Type":"ContainerStarted","Data":"bda0d8bb0311e82bb43d40db54070ce8e75c1d2e49bce76fd3b99040e3ff8b90"}
Jul 15 20:03:33.352624 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:33.352581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn" event={"ID":"b4705a9b-7f00-47a1-8144-d88e3b97f8ed","Type":"ContainerStarted","Data":"6f1dfb53d9634bf00d916aef55d8de51b8ff9f85963461c70b96e0b799047d98"}
Jul 15 20:03:33.376840 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:33.376796    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8h7tn" podStartSLOduration=2.090786032 podStartE2EDuration="25.37678169s" totalImagesPullingTime="23.285995658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:33.375785719 +0000 UTC m=+546.633187900" watchObservedRunningTime="2026-07-15 20:03:33.37678169 +0000 UTC m=+546.634183926"
Jul 15 20:03:52.515348 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.515309    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-47csr"]
Jul 15 20:03:52.558626 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.558588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-47csr"]
Jul 15 20:03:52.558853 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.558682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-47csr"
Jul 15 20:03:52.561609 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.561590    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n9w2g\""
Jul 15 20:03:52.609424 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.609393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4ssx\" (UniqueName: \"kubernetes.io/projected/067ff5b5-ec4e-4879-a342-9723b017c6c5-kube-api-access-z4ssx\") pod \"authorino-58cd955675-47csr\" (UID: \"067ff5b5-ec4e-4879-a342-9723b017c6c5\") " pod="kuadrant-system/authorino-58cd955675-47csr"
Jul 15 20:03:52.709746 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.709683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4ssx\" (UniqueName: \"kubernetes.io/projected/067ff5b5-ec4e-4879-a342-9723b017c6c5-kube-api-access-z4ssx\") pod \"authorino-58cd955675-47csr\" (UID: \"067ff5b5-ec4e-4879-a342-9723b017c6c5\") " pod="kuadrant-system/authorino-58cd955675-47csr"
Jul 15 20:03:52.721462 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.721438    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4ssx\" (UniqueName: \"kubernetes.io/projected/067ff5b5-ec4e-4879-a342-9723b017c6c5-kube-api-access-z4ssx\") pod \"authorino-58cd955675-47csr\" (UID: \"067ff5b5-ec4e-4879-a342-9723b017c6c5\") " pod="kuadrant-system/authorino-58cd955675-47csr"
Jul 15 20:03:52.758948 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.758920    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mkzb"]
Jul 15 20:03:52.784199 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.784141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mkzb"]
Jul 15 20:03:52.784299 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.784224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mkzb"
Jul 15 20:03:52.810275 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.810249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjq5f\" (UniqueName: \"kubernetes.io/projected/e1a9679b-887f-4f14-b294-b6f9a5231c3e-kube-api-access-jjq5f\") pod \"authorino-75d556bd77-7mkzb\" (UID: \"e1a9679b-887f-4f14-b294-b6f9a5231c3e\") " pod="kuadrant-system/authorino-75d556bd77-7mkzb"
Jul 15 20:03:52.867260 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.867232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-47csr"
Jul 15 20:03:52.911678 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.911648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjq5f\" (UniqueName: \"kubernetes.io/projected/e1a9679b-887f-4f14-b294-b6f9a5231c3e-kube-api-access-jjq5f\") pod \"authorino-75d556bd77-7mkzb\" (UID: \"e1a9679b-887f-4f14-b294-b6f9a5231c3e\") " pod="kuadrant-system/authorino-75d556bd77-7mkzb"
Jul 15 20:03:52.921824 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:52.921801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjq5f\" (UniqueName: \"kubernetes.io/projected/e1a9679b-887f-4f14-b294-b6f9a5231c3e-kube-api-access-jjq5f\") pod \"authorino-75d556bd77-7mkzb\" (UID: \"e1a9679b-887f-4f14-b294-b6f9a5231c3e\") " pod="kuadrant-system/authorino-75d556bd77-7mkzb"
Jul 15 20:03:53.023518 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:53.023482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-47csr"]
Jul 15 20:03:53.026488 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:03:53.026451    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod067ff5b5_ec4e_4879_a342_9723b017c6c5.slice/crio-45f6f5a55ae2c09b5d70f04985ef4b9e861f2306887639b7a61e4e0a358242d4 WatchSource:0}: Error finding container 45f6f5a55ae2c09b5d70f04985ef4b9e861f2306887639b7a61e4e0a358242d4: Status 404 returned error can't find the container with id 45f6f5a55ae2c09b5d70f04985ef4b9e861f2306887639b7a61e4e0a358242d4
Jul 15 20:03:53.092798 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:53.092705    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mkzb"
Jul 15 20:03:53.247413 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:53.247378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mkzb"]
Jul 15 20:03:53.250509 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:03:53.250486    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1a9679b_887f_4f14_b294_b6f9a5231c3e.slice/crio-9ed1e3eb344ca9c5b8cdff8f4b1968e5b76557a30f1d3ba440802e95627839c9 WatchSource:0}: Error finding container 9ed1e3eb344ca9c5b8cdff8f4b1968e5b76557a30f1d3ba440802e95627839c9: Status 404 returned error can't find the container with id 9ed1e3eb344ca9c5b8cdff8f4b1968e5b76557a30f1d3ba440802e95627839c9
Jul 15 20:03:53.422988 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:53.422951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-47csr" event={"ID":"067ff5b5-ec4e-4879-a342-9723b017c6c5","Type":"ContainerStarted","Data":"45f6f5a55ae2c09b5d70f04985ef4b9e861f2306887639b7a61e4e0a358242d4"}
Jul 15 20:03:53.424089 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:53.424053    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mkzb" event={"ID":"e1a9679b-887f-4f14-b294-b6f9a5231c3e","Type":"ContainerStarted","Data":"9ed1e3eb344ca9c5b8cdff8f4b1968e5b76557a30f1d3ba440802e95627839c9"}
Jul 15 20:03:58.450524 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:58.450486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-47csr" event={"ID":"067ff5b5-ec4e-4879-a342-9723b017c6c5","Type":"ContainerStarted","Data":"955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8"}
Jul 15 20:03:58.451978 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:58.451956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mkzb" event={"ID":"e1a9679b-887f-4f14-b294-b6f9a5231c3e","Type":"ContainerStarted","Data":"2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020"}
Jul 15 20:03:58.473736 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:58.473674    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-47csr" podStartSLOduration=1.628408646 podStartE2EDuration="6.473657035s" totalImagesPullingTime="4.845248389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:58.472930691 +0000 UTC m=+571.730332876" watchObservedRunningTime="2026-07-15 20:03:58.473657035 +0000 UTC m=+571.731059222"
Jul 15 20:03:58.490309 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:58.490268    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7mkzb" podStartSLOduration=1.857363473 podStartE2EDuration="6.490252512s" totalImagesPullingTime="4.632889039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:58.489866265 +0000 UTC m=+571.747268446" watchObservedRunningTime="2026-07-15 20:03:58.490252512 +0000 UTC m=+571.747654689"
Jul 15 20:03:58.517416 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:03:58.517387    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-47csr"]
Jul 15 20:04:00.460182 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:00.460145    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-47csr" podUID="067ff5b5-ec4e-4879-a342-9723b017c6c5" containerName="authorino" containerID="cri-o://955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8" gracePeriod=30
Jul 15 20:04:00.744181 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:00.744149    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-47csr"
Jul 15 20:04:00.868960 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:00.868926    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4ssx\" (UniqueName: \"kubernetes.io/projected/067ff5b5-ec4e-4879-a342-9723b017c6c5-kube-api-access-z4ssx\") pod \"067ff5b5-ec4e-4879-a342-9723b017c6c5\" (UID: \"067ff5b5-ec4e-4879-a342-9723b017c6c5\") "
Jul 15 20:04:00.871259 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:00.871238    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/067ff5b5-ec4e-4879-a342-9723b017c6c5-kube-api-access-z4ssx" (OuterVolumeSpecName: "kube-api-access-z4ssx") pod "067ff5b5-ec4e-4879-a342-9723b017c6c5" (UID: "067ff5b5-ec4e-4879-a342-9723b017c6c5"). InnerVolumeSpecName "kube-api-access-z4ssx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:04:00.970142 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:00.970080    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4ssx\" (UniqueName: \"kubernetes.io/projected/067ff5b5-ec4e-4879-a342-9723b017c6c5-kube-api-access-z4ssx\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:04:01.465233 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.465195    2563 generic.go:356] "Generic (PLEG): container finished" podID="067ff5b5-ec4e-4879-a342-9723b017c6c5" containerID="955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8" exitCode=0
Jul 15 20:04:01.465658 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.465242    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-47csr"
Jul 15 20:04:01.465658 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.465252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-47csr" event={"ID":"067ff5b5-ec4e-4879-a342-9723b017c6c5","Type":"ContainerDied","Data":"955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8"}
Jul 15 20:04:01.465658 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.465283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-47csr" event={"ID":"067ff5b5-ec4e-4879-a342-9723b017c6c5","Type":"ContainerDied","Data":"45f6f5a55ae2c09b5d70f04985ef4b9e861f2306887639b7a61e4e0a358242d4"}
Jul 15 20:04:01.465658 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.465298    2563 scope.go:117] "RemoveContainer" containerID="955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8"
Jul 15 20:04:01.473024 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.473003    2563 scope.go:117] "RemoveContainer" containerID="955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8"
Jul 15 20:04:01.473360 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:04:01.473332    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8\": container with ID starting with 955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8 not found: ID does not exist" containerID="955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8"
Jul 15 20:04:01.473464 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.473367    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8"} err="failed to get container status \"955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8\": rpc error: code = NotFound desc = could not find container \"955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8\": container with ID starting with 955605973d4f0b12b66fa306a564e6a2697afbfe8659370201724bf47951d9d8 not found: ID does not exist"
Jul 15 20:04:01.483910 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.483889    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-47csr"]
Jul 15 20:04:01.485127 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:01.485109    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-47csr"]
Jul 15 20:04:03.297012 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:03.296982    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="067ff5b5-ec4e-4879-a342-9723b017c6c5" path="/var/lib/kubelet/pods/067ff5b5-ec4e-4879-a342-9723b017c6c5/volumes"
Jul 15 20:04:20.073287 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.073251    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdzn5"]
Jul 15 20:04:20.073660 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.073512    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="067ff5b5-ec4e-4879-a342-9723b017c6c5" containerName="authorino"
Jul 15 20:04:20.073660 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.073522    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="067ff5b5-ec4e-4879-a342-9723b017c6c5" containerName="authorino"
Jul 15 20:04:20.073660 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.073603    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="067ff5b5-ec4e-4879-a342-9723b017c6c5" containerName="authorino"
Jul 15 20:04:20.075318 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.075301    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdzn5"
Jul 15 20:04:20.093140 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.093113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdzn5"]
Jul 15 20:04:20.105846 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.105815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk946\" (UniqueName: \"kubernetes.io/projected/fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63-kube-api-access-xk946\") pod \"authorino-5d4b67c446-sdzn5\" (UID: \"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63\") " pod="kuadrant-system/authorino-5d4b67c446-sdzn5"
Jul 15 20:04:20.206940 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.206907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk946\" (UniqueName: \"kubernetes.io/projected/fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63-kube-api-access-xk946\") pod \"authorino-5d4b67c446-sdzn5\" (UID: \"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63\") " pod="kuadrant-system/authorino-5d4b67c446-sdzn5"
Jul 15 20:04:20.217104 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.217070    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk946\" (UniqueName: \"kubernetes.io/projected/fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63-kube-api-access-xk946\") pod \"authorino-5d4b67c446-sdzn5\" (UID: \"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63\") " pod="kuadrant-system/authorino-5d4b67c446-sdzn5"
Jul 15 20:04:20.322153 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.322121    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdzn5"]
Jul 15 20:04:20.322305 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.322276    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdzn5"
Jul 15 20:04:20.389216 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.389119    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66cd979bcd-677tt"]
Jul 15 20:04:20.393793 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.393762    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd979bcd-677tt"
Jul 15 20:04:20.449473 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.449443    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66cd979bcd-677tt"]
Jul 15 20:04:20.457428 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.457403    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66cd979bcd-677tt"]
Jul 15 20:04:20.457676 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:04:20.457655    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4vrj7], unattached volumes=[], failed to process volumes=[kube-api-access-4vrj7]: context canceled" pod="kuadrant-system/authorino-66cd979bcd-677tt" podUID="b95db002-0e4b-46df-b07a-88f835a1ec11"
Jul 15 20:04:20.472753 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.472686    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdzn5"]
Jul 15 20:04:20.475409 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:04:20.475383    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb1eb6d6_a3cb_426b_9d87_cf01d0a8ab63.slice/crio-926603fc4b68c0a28fa546e2babeace49395bab0560a5fc1ff689cef6fc15819 WatchSource:0}: Error finding container 926603fc4b68c0a28fa546e2babeace49395bab0560a5fc1ff689cef6fc15819: Status 404 returned error can't find the container with id 926603fc4b68c0a28fa546e2babeace49395bab0560a5fc1ff689cef6fc15819
Jul 15 20:04:20.491126 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.491106    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65d5dc4f7d-8qxlg"]
Jul 15 20:04:20.493161 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.493145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.495880 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.495861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 15 20:04:20.508978 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.508955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vrj7\" (UniqueName: \"kubernetes.io/projected/b95db002-0e4b-46df-b07a-88f835a1ec11-kube-api-access-4vrj7\") pod \"authorino-66cd979bcd-677tt\" (UID: \"b95db002-0e4b-46df-b07a-88f835a1ec11\") " pod="kuadrant-system/authorino-66cd979bcd-677tt"
Jul 15 20:04:20.508978 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.508964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d5dc4f7d-8qxlg"]
Jul 15 20:04:20.541704 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.541683    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd979bcd-677tt"
Jul 15 20:04:20.541830 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.541678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sdzn5" event={"ID":"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63","Type":"ContainerStarted","Data":"926603fc4b68c0a28fa546e2babeace49395bab0560a5fc1ff689cef6fc15819"}
Jul 15 20:04:20.546876 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.546859    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd979bcd-677tt"
Jul 15 20:04:20.610093 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.610072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cnzd\" (UniqueName: \"kubernetes.io/projected/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-kube-api-access-9cnzd\") pod \"authorino-65d5dc4f7d-8qxlg\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") " pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.610204 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.610110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-tls-cert\") pod \"authorino-65d5dc4f7d-8qxlg\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") " pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.610204 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.610142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vrj7\" (UniqueName: \"kubernetes.io/projected/b95db002-0e4b-46df-b07a-88f835a1ec11-kube-api-access-4vrj7\") pod \"authorino-66cd979bcd-677tt\" (UID: \"b95db002-0e4b-46df-b07a-88f835a1ec11\") " pod="kuadrant-system/authorino-66cd979bcd-677tt"
Jul 15 20:04:20.618843 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.618816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vrj7\" (UniqueName: \"kubernetes.io/projected/b95db002-0e4b-46df-b07a-88f835a1ec11-kube-api-access-4vrj7\") pod \"authorino-66cd979bcd-677tt\" (UID: \"b95db002-0e4b-46df-b07a-88f835a1ec11\") " pod="kuadrant-system/authorino-66cd979bcd-677tt"
Jul 15 20:04:20.710887 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.710859    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vrj7\" (UniqueName: \"kubernetes.io/projected/b95db002-0e4b-46df-b07a-88f835a1ec11-kube-api-access-4vrj7\") pod \"b95db002-0e4b-46df-b07a-88f835a1ec11\" (UID: \"b95db002-0e4b-46df-b07a-88f835a1ec11\") "
Jul 15 20:04:20.711021 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.710966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-tls-cert\") pod \"authorino-65d5dc4f7d-8qxlg\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") " pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.711065 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.711019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cnzd\" (UniqueName: \"kubernetes.io/projected/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-kube-api-access-9cnzd\") pod \"authorino-65d5dc4f7d-8qxlg\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") " pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.713118 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.713089    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b95db002-0e4b-46df-b07a-88f835a1ec11-kube-api-access-4vrj7" (OuterVolumeSpecName: "kube-api-access-4vrj7") pod "b95db002-0e4b-46df-b07a-88f835a1ec11" (UID: "b95db002-0e4b-46df-b07a-88f835a1ec11"). InnerVolumeSpecName "kube-api-access-4vrj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:04:20.713417 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.713400    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-tls-cert\") pod \"authorino-65d5dc4f7d-8qxlg\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") " pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.720644 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.720626    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cnzd\" (UniqueName: \"kubernetes.io/projected/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-kube-api-access-9cnzd\") pod \"authorino-65d5dc4f7d-8qxlg\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") " pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.802280 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.802261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:04:20.812201 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.812181    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vrj7\" (UniqueName: \"kubernetes.io/projected/b95db002-0e4b-46df-b07a-88f835a1ec11-kube-api-access-4vrj7\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:04:20.932681 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:20.932650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d5dc4f7d-8qxlg"]
Jul 15 20:04:20.935525 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:04:20.935504    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod639ddd84_354a_43bc_a0d6_a90b7d8e92c6.slice/crio-2877940a72ef58293c9f3d9512c64664bd0a1c2d9a2761ede29fc862dab9cc37 WatchSource:0}: Error finding container 2877940a72ef58293c9f3d9512c64664bd0a1c2d9a2761ede29fc862dab9cc37: Status 404 returned error can't find the container with id 2877940a72ef58293c9f3d9512c64664bd0a1c2d9a2761ede29fc862dab9cc37
Jul 15 20:04:21.546778 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.546690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sdzn5" event={"ID":"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63","Type":"ContainerStarted","Data":"9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a"}
Jul 15 20:04:21.547192 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.546814    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-sdzn5" podUID="fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63" containerName="authorino" containerID="cri-o://9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a" gracePeriod=30
Jul 15 20:04:21.548455 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.548429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg" event={"ID":"639ddd84-354a-43bc-a0d6-a90b7d8e92c6","Type":"ContainerStarted","Data":"4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50"}
Jul 15 20:04:21.548575 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.548467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg" event={"ID":"639ddd84-354a-43bc-a0d6-a90b7d8e92c6","Type":"ContainerStarted","Data":"2877940a72ef58293c9f3d9512c64664bd0a1c2d9a2761ede29fc862dab9cc37"}
Jul 15 20:04:21.548575 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.548469    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd979bcd-677tt"
Jul 15 20:04:21.575012 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.574973    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-sdzn5" podStartSLOduration=1.160283409 podStartE2EDuration="1.574961243s" totalImagesPullingTime="414.677834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:04:21.574857285 +0000 UTC m=+594.832259463" watchObservedRunningTime="2026-07-15 20:04:21.574961243 +0000 UTC m=+594.832363422"
Jul 15 20:04:21.624362 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.624337    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66cd979bcd-677tt"]
Jul 15 20:04:21.629245 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.629223    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66cd979bcd-677tt"]
Jul 15 20:04:21.714392 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.714332    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg" podStartSLOduration=1.36392796 podStartE2EDuration="1.714310183s" totalImagesPullingTime="350.382223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:04:21.652039401 +0000 UTC m=+594.909441581" watchObservedRunningTime="2026-07-15 20:04:21.714310183 +0000 UTC m=+594.971712362"
Jul 15 20:04:21.714933 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.714814    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mkzb"]
Jul 15 20:04:21.715018 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.714996    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7mkzb" podUID="e1a9679b-887f-4f14-b294-b6f9a5231c3e" containerName="authorino" containerID="cri-o://2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020" gracePeriod=30
Jul 15 20:04:21.828182 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.828156    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdzn5"
Jul 15 20:04:21.921984 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.921953    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk946\" (UniqueName: \"kubernetes.io/projected/fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63-kube-api-access-xk946\") pod \"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63\" (UID: \"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63\") "
Jul 15 20:04:21.924089 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.924064    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63-kube-api-access-xk946" (OuterVolumeSpecName: "kube-api-access-xk946") pod "fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63" (UID: "fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63"). InnerVolumeSpecName "kube-api-access-xk946". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:04:21.960418 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:21.960395    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mkzb"
Jul 15 20:04:22.023107 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.023079    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjq5f\" (UniqueName: \"kubernetes.io/projected/e1a9679b-887f-4f14-b294-b6f9a5231c3e-kube-api-access-jjq5f\") pod \"e1a9679b-887f-4f14-b294-b6f9a5231c3e\" (UID: \"e1a9679b-887f-4f14-b294-b6f9a5231c3e\") "
Jul 15 20:04:22.023264 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.023211    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk946\" (UniqueName: \"kubernetes.io/projected/fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63-kube-api-access-xk946\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:04:22.025204 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.025182    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1a9679b-887f-4f14-b294-b6f9a5231c3e-kube-api-access-jjq5f" (OuterVolumeSpecName: "kube-api-access-jjq5f") pod "e1a9679b-887f-4f14-b294-b6f9a5231c3e" (UID: "e1a9679b-887f-4f14-b294-b6f9a5231c3e"). InnerVolumeSpecName "kube-api-access-jjq5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:04:22.124323 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.124285    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjq5f\" (UniqueName: \"kubernetes.io/projected/e1a9679b-887f-4f14-b294-b6f9a5231c3e-kube-api-access-jjq5f\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:04:22.553398 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.553312    2563 generic.go:356] "Generic (PLEG): container finished" podID="e1a9679b-887f-4f14-b294-b6f9a5231c3e" containerID="2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020" exitCode=0
Jul 15 20:04:22.553398 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.553352    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mkzb"
Jul 15 20:04:22.553920 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.553394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mkzb" event={"ID":"e1a9679b-887f-4f14-b294-b6f9a5231c3e","Type":"ContainerDied","Data":"2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020"}
Jul 15 20:04:22.553920 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.553432    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mkzb" event={"ID":"e1a9679b-887f-4f14-b294-b6f9a5231c3e","Type":"ContainerDied","Data":"9ed1e3eb344ca9c5b8cdff8f4b1968e5b76557a30f1d3ba440802e95627839c9"}
Jul 15 20:04:22.553920 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.553454    2563 scope.go:117] "RemoveContainer" containerID="2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020"
Jul 15 20:04:22.554873 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.554676    2563 generic.go:356] "Generic (PLEG): container finished" podID="fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63" containerID="9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a" exitCode=0
Jul 15 20:04:22.554873 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.554742    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdzn5"
Jul 15 20:04:22.554873 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.554758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sdzn5" event={"ID":"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63","Type":"ContainerDied","Data":"9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a"}
Jul 15 20:04:22.554873 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.554792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sdzn5" event={"ID":"fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63","Type":"ContainerDied","Data":"926603fc4b68c0a28fa546e2babeace49395bab0560a5fc1ff689cef6fc15819"}
Jul 15 20:04:22.561872 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.561843    2563 scope.go:117] "RemoveContainer" containerID="2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020"
Jul 15 20:04:22.562196 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:04:22.562128    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020\": container with ID starting with 2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020 not found: ID does not exist" containerID="2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020"
Jul 15 20:04:22.562196 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.562162    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020"} err="failed to get container status \"2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020\": rpc error: code = NotFound desc = could not find container \"2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020\": container with ID starting with 2d762edfd35bc33c089af7530962ceb215483f45db6592a1409fda9ffbff0020 not found: ID does not exist"
Jul 15 20:04:22.562196 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.562185    2563 scope.go:117] "RemoveContainer" containerID="9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a"
Jul 15 20:04:22.569384 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.569367    2563 scope.go:117] "RemoveContainer" containerID="9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a"
Jul 15 20:04:22.569621 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:04:22.569604    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a\": container with ID starting with 9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a not found: ID does not exist" containerID="9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a"
Jul 15 20:04:22.569673 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.569629    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a"} err="failed to get container status \"9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a\": rpc error: code = NotFound desc = could not find container \"9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a\": container with ID starting with 9b16e2afd231b0e1cb526c8349b01e20ef9bf92bea532678234aac06f09aae0a not found: ID does not exist"
Jul 15 20:04:22.576482 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.576457    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mkzb"]
Jul 15 20:04:22.581091 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.581074    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mkzb"]
Jul 15 20:04:22.591997 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.591976    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdzn5"]
Jul 15 20:04:22.595994 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:22.595976    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdzn5"]
Jul 15 20:04:23.296644 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:23.296613    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b95db002-0e4b-46df-b07a-88f835a1ec11" path="/var/lib/kubelet/pods/b95db002-0e4b-46df-b07a-88f835a1ec11/volumes"
Jul 15 20:04:23.296860 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:23.296846    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1a9679b-887f-4f14-b294-b6f9a5231c3e" path="/var/lib/kubelet/pods/e1a9679b-887f-4f14-b294-b6f9a5231c3e/volumes"
Jul 15 20:04:23.297121 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:23.297110    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63" path="/var/lib/kubelet/pods/fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63/volumes"
Jul 15 20:04:26.378431 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.378399    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-767bd6f858-f4g4f"]
Jul 15 20:04:26.378846 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.378698    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63" containerName="authorino"
Jul 15 20:04:26.378846 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.378711    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63" containerName="authorino"
Jul 15 20:04:26.378846 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.378741    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1a9679b-887f-4f14-b294-b6f9a5231c3e" containerName="authorino"
Jul 15 20:04:26.378846 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.378748    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1a9679b-887f-4f14-b294-b6f9a5231c3e" containerName="authorino"
Jul 15 20:04:26.378846 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.378795    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1a9679b-887f-4f14-b294-b6f9a5231c3e" containerName="authorino"
Jul 15 20:04:26.378846 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.378803    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1eb6d6-a3cb-426b-9d87-cf01d0a8ab63" containerName="authorino"
Jul 15 20:04:26.383138 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.383118    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.386013 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.385991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 20:04:26.386137 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.386091    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8qvkx\""
Jul 15 20:04:26.386329 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.386314    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 20:04:26.396575 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.396554    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-767bd6f858-f4g4f"]
Jul 15 20:04:26.456672 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.456637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c226800f-038e-4e0b-a717-8b0e4e20dbe7-webhook-cert\") pod \"maas-controller-767bd6f858-f4g4f\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") " pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.456816 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.456751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42gv6\" (UniqueName: \"kubernetes.io/projected/c226800f-038e-4e0b-a717-8b0e4e20dbe7-kube-api-access-42gv6\") pod \"maas-controller-767bd6f858-f4g4f\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") " pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.557495 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.557468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42gv6\" (UniqueName: \"kubernetes.io/projected/c226800f-038e-4e0b-a717-8b0e4e20dbe7-kube-api-access-42gv6\") pod \"maas-controller-767bd6f858-f4g4f\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") " pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.557655 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.557511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c226800f-038e-4e0b-a717-8b0e4e20dbe7-webhook-cert\") pod \"maas-controller-767bd6f858-f4g4f\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") " pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.559948 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.559922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c226800f-038e-4e0b-a717-8b0e4e20dbe7-webhook-cert\") pod \"maas-controller-767bd6f858-f4g4f\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") " pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.568022 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.568000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42gv6\" (UniqueName: \"kubernetes.io/projected/c226800f-038e-4e0b-a717-8b0e4e20dbe7-kube-api-access-42gv6\") pod \"maas-controller-767bd6f858-f4g4f\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") " pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.694033 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.693941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:26.829537 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:26.829508    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-767bd6f858-f4g4f"]
Jul 15 20:04:26.832088 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:04:26.832067    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc226800f_038e_4e0b_a717_8b0e4e20dbe7.slice/crio-0c4bb777f4fcaa1d97c2236ccce6e916fe0d2b8c455947772f7309896703882f WatchSource:0}: Error finding container 0c4bb777f4fcaa1d97c2236ccce6e916fe0d2b8c455947772f7309896703882f: Status 404 returned error can't find the container with id 0c4bb777f4fcaa1d97c2236ccce6e916fe0d2b8c455947772f7309896703882f
Jul 15 20:04:27.138478 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:27.138449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:04:27.139901 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:27.139881    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:04:27.145242 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:27.145221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:04:27.146867 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:27.146851    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:04:27.584760 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:27.584659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-767bd6f858-f4g4f" event={"ID":"c226800f-038e-4e0b-a717-8b0e4e20dbe7","Type":"ContainerStarted","Data":"0c4bb777f4fcaa1d97c2236ccce6e916fe0d2b8c455947772f7309896703882f"}
Jul 15 20:04:29.349118 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:29.349096    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 20:04:29.595495 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:29.595407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-767bd6f858-f4g4f" event={"ID":"c226800f-038e-4e0b-a717-8b0e4e20dbe7","Type":"ContainerStarted","Data":"eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0"}
Jul 15 20:04:29.595636 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:29.595548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:29.622929 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:29.622881    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-767bd6f858-f4g4f" podStartSLOduration=1.110432718 podStartE2EDuration="3.622866812s" totalImagesPullingTime="2.512434094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:04:29.622039018 +0000 UTC m=+602.879441198" watchObservedRunningTime="2026-07-15 20:04:29.622866812 +0000 UTC m=+602.880268990"
Jul 15 20:04:40.602066 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:40.602033    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:41.642041 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:41.642006    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-767bd6f858-f4g4f"]
Jul 15 20:04:41.642468 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:41.642227    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-767bd6f858-f4g4f" podUID="c226800f-038e-4e0b-a717-8b0e4e20dbe7" containerName="manager" containerID="cri-o://eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0" gracePeriod=30
Jul 15 20:04:41.884380 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:41.884359    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:41.982012 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:41.981937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42gv6\" (UniqueName: \"kubernetes.io/projected/c226800f-038e-4e0b-a717-8b0e4e20dbe7-kube-api-access-42gv6\") pod \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") "
Jul 15 20:04:41.982012 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:41.981993    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c226800f-038e-4e0b-a717-8b0e4e20dbe7-webhook-cert\") pod \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\" (UID: \"c226800f-038e-4e0b-a717-8b0e4e20dbe7\") "
Jul 15 20:04:41.984312 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:41.984278    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c226800f-038e-4e0b-a717-8b0e4e20dbe7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c226800f-038e-4e0b-a717-8b0e4e20dbe7" (UID: "c226800f-038e-4e0b-a717-8b0e4e20dbe7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:04:41.984428 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:41.984317    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c226800f-038e-4e0b-a717-8b0e4e20dbe7-kube-api-access-42gv6" (OuterVolumeSpecName: "kube-api-access-42gv6") pod "c226800f-038e-4e0b-a717-8b0e4e20dbe7" (UID: "c226800f-038e-4e0b-a717-8b0e4e20dbe7"). InnerVolumeSpecName "kube-api-access-42gv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:04:42.083082 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.083052    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42gv6\" (UniqueName: \"kubernetes.io/projected/c226800f-038e-4e0b-a717-8b0e4e20dbe7-kube-api-access-42gv6\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:04:42.083082 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.083078    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c226800f-038e-4e0b-a717-8b0e4e20dbe7-webhook-cert\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:04:42.655248 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.655212    2563 generic.go:356] "Generic (PLEG): container finished" podID="c226800f-038e-4e0b-a717-8b0e4e20dbe7" containerID="eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0" exitCode=0
Jul 15 20:04:42.655641 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.655259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-767bd6f858-f4g4f" event={"ID":"c226800f-038e-4e0b-a717-8b0e4e20dbe7","Type":"ContainerDied","Data":"eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0"}
Jul 15 20:04:42.655641 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.655322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-767bd6f858-f4g4f" event={"ID":"c226800f-038e-4e0b-a717-8b0e4e20dbe7","Type":"ContainerDied","Data":"0c4bb777f4fcaa1d97c2236ccce6e916fe0d2b8c455947772f7309896703882f"}
Jul 15 20:04:42.655641 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.655260    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-767bd6f858-f4g4f"
Jul 15 20:04:42.655641 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.655349    2563 scope.go:117] "RemoveContainer" containerID="eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0"
Jul 15 20:04:42.663823 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.663807    2563 scope.go:117] "RemoveContainer" containerID="eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0"
Jul 15 20:04:42.664084 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:04:42.664067    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0\": container with ID starting with eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0 not found: ID does not exist" containerID="eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0"
Jul 15 20:04:42.664150 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.664090    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0"} err="failed to get container status \"eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0\": rpc error: code = NotFound desc = could not find container \"eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0\": container with ID starting with eca35489a0c2c498ad7bb7cd5e9e7c86f4494ec69861db273f68bf1c114430b0 not found: ID does not exist"
Jul 15 20:04:42.675902 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.675880    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-767bd6f858-f4g4f"]
Jul 15 20:04:42.679890 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:42.679870    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-767bd6f858-f4g4f"]
Jul 15 20:04:43.296578 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:04:43.296547    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c226800f-038e-4e0b-a717-8b0e4e20dbe7" path="/var/lib/kubelet/pods/c226800f-038e-4e0b-a717-8b0e4e20dbe7/volumes"
Jul 15 20:05:30.592211 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.592181    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"]
Jul 15 20:05:30.592606 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.592474    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c226800f-038e-4e0b-a717-8b0e4e20dbe7" containerName="manager"
Jul 15 20:05:30.592606 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.592486    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c226800f-038e-4e0b-a717-8b0e4e20dbe7" containerName="manager"
Jul 15 20:05:30.592606 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.592557    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c226800f-038e-4e0b-a717-8b0e4e20dbe7" containerName="manager"
Jul 15 20:05:30.595202 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.595183    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.598390 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.598371    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 20:05:30.598631 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.598614    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9fcf5\""
Jul 15 20:05:30.599264 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.599243    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 15 20:05:30.599355 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.599274    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 20:05:30.610098 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.610077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"]
Jul 15 20:05:30.638297 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.638265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnl9f\" (UniqueName: \"kubernetes.io/projected/a88c83dc-5f14-449e-bc75-f9fdd51537e6-kube-api-access-mnl9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.638418 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.638306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a88c83dc-5f14-449e-bc75-f9fdd51537e6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.638418 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.638346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.638418 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.638381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.638418 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.638416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.638622 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.638440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.738996 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.738963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnl9f\" (UniqueName: \"kubernetes.io/projected/a88c83dc-5f14-449e-bc75-f9fdd51537e6-kube-api-access-mnl9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739145 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a88c83dc-5f14-449e-bc75-f9fdd51537e6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739145 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739145 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739263 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739307 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739492 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739492 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.739648 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.739568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.741438 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.741408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a88c83dc-5f14-449e-bc75-f9fdd51537e6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.741830 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.741814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a88c83dc-5f14-449e-bc75-f9fdd51537e6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.747118 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.747094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnl9f\" (UniqueName: \"kubernetes.io/projected/a88c83dc-5f14-449e-bc75-f9fdd51537e6-kube-api-access-mnl9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw\" (UID: \"a88c83dc-5f14-449e-bc75-f9fdd51537e6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:30.905453 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:30.905428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:31.040788 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:31.040746    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"]
Jul 15 20:05:31.046551 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:31.046531    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:05:31.846595 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:31.846556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw" event={"ID":"a88c83dc-5f14-449e-bc75-f9fdd51537e6","Type":"ContainerStarted","Data":"10a3a7589a680debaccb96afbe06344f7f6efeb5b87af73aad953ee2a59995eb"}
Jul 15 20:05:32.851654 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:32.851623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw" event={"ID":"a88c83dc-5f14-449e-bc75-f9fdd51537e6","Type":"ContainerStarted","Data":"71a8bfcf08e0e524edf2654fdaa1d49970e79b38c31b9f58d3974d2155f7e38e"}
Jul 15 20:05:32.874408 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:32.874356    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw" podStartSLOduration=1.71419448 podStartE2EDuration="2.874341036s" totalImagesPullingTime="1.160146556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:32.873872376 +0000 UTC m=+666.131274556" watchObservedRunningTime="2026-07-15 20:05:32.874341036 +0000 UTC m=+666.131743215"
Jul 15 20:05:40.906428 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:40.906391    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:40.906932 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:40.906456    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:40.919283 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:40.919255    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:05:41.901310 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:05:41.901277    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw"
Jul 15 20:06:04.773644 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.773600    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"]
Jul 15 20:06:04.779511 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.779489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:04.782130 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.782113    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 15 20:06:04.787884 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.787858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"]
Jul 15 20:06:04.932599 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.932564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6gbp\" (UniqueName: \"kubernetes.io/projected/77c07a00-1582-47f5-8425-21b5c05817a4-kube-api-access-p6gbp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:04.932796 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.932607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:04.932796 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.932628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:04.932796 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.932755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c07a00-1582-47f5-8425-21b5c05817a4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:04.932796 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.932778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:04.932939 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:04.932809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.033781 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.033654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c07a00-1582-47f5-8425-21b5c05817a4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.033781 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.033703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.033781 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.033770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.034065 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.033801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6gbp\" (UniqueName: \"kubernetes.io/projected/77c07a00-1582-47f5-8425-21b5c05817a4-kube-api-access-p6gbp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.034065 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.033868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.034065 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.033911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.034248 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.034225    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.034309 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.034286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.034345 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.034293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.036226 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.036205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c07a00-1582-47f5-8425-21b5c05817a4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.036449 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.036433    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c07a00-1582-47f5-8425-21b5c05817a4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.042899 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.042865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6gbp\" (UniqueName: \"kubernetes.io/projected/77c07a00-1582-47f5-8425-21b5c05817a4-kube-api-access-p6gbp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws\" (UID: \"77c07a00-1582-47f5-8425-21b5c05817a4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.089916 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.089885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:05.225081 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.225046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"]
Jul 15 20:06:05.227971 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:06:05.227940    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c07a00_1582_47f5_8425_21b5c05817a4.slice/crio-127b06528ef64eb83751e7f38df758c5892b6527ba5cf10baa57bf32bc3e7d9e WatchSource:0}: Error finding container 127b06528ef64eb83751e7f38df758c5892b6527ba5cf10baa57bf32bc3e7d9e: Status 404 returned error can't find the container with id 127b06528ef64eb83751e7f38df758c5892b6527ba5cf10baa57bf32bc3e7d9e
Jul 15 20:06:05.985123 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.985083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws" event={"ID":"77c07a00-1582-47f5-8425-21b5c05817a4","Type":"ContainerStarted","Data":"8726873a5e4bab512a7a6d64fcb1e8511251d3be6302be5e7b177a048f8bd905"}
Jul 15 20:06:05.985559 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:05.985162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws" event={"ID":"77c07a00-1582-47f5-8425-21b5c05817a4","Type":"ContainerStarted","Data":"127b06528ef64eb83751e7f38df758c5892b6527ba5cf10baa57bf32bc3e7d9e"}
Jul 15 20:06:06.006978 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:06.006928    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws" podStartSLOduration=1.680402925 podStartE2EDuration="2.006913261s" totalImagesPullingTime="326.510336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:06:06.005798596 +0000 UTC m=+699.263200776" watchObservedRunningTime="2026-07-15 20:06:06.006913261 +0000 UTC m=+699.264315440"
Jul 15 20:06:15.090484 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:15.090447    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:15.091009 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:15.090522    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:15.102835 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:15.102813    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:16.038604 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:16.038569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws"
Jul 15 20:06:17.581215 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.581180    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"]
Jul 15 20:06:17.586416 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.586390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.589244 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.589216    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 15 20:06:17.594286 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.594263    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"]
Jul 15 20:06:17.650700 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.650669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.650700 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.650702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.650919 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.650744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.650919 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.650837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.650919 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.650870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dznqd\" (UniqueName: \"kubernetes.io/projected/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-kube-api-access-dznqd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.651034 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.650922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.751566 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.751528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.751765 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.751585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.751765 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.751613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.751765 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.751637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.751765 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.751700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.751765 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.751759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dznqd\" (UniqueName: \"kubernetes.io/projected/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-kube-api-access-dznqd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.752080 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.752051    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.752195 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.752064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.752195 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.752152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.754079 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.754055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.754274 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.754258    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.762803 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.762754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dznqd\" (UniqueName: \"kubernetes.io/projected/fb8a1abb-7071-4712-82a9-b1ba1a96ba53-kube-api-access-dznqd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs\" (UID: \"fb8a1abb-7071-4712-82a9-b1ba1a96ba53\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:17.896298 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:17.896268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:18.027852 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:18.027826    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"]
Jul 15 20:06:18.030341 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:06:18.030314    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb8a1abb_7071_4712_82a9_b1ba1a96ba53.slice/crio-c3e6e88f4dac4dbd7079e877f0aedccd47044adcada01b8cf821612c07221c10 WatchSource:0}: Error finding container c3e6e88f4dac4dbd7079e877f0aedccd47044adcada01b8cf821612c07221c10: Status 404 returned error can't find the container with id c3e6e88f4dac4dbd7079e877f0aedccd47044adcada01b8cf821612c07221c10
Jul 15 20:06:19.039302 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:19.039267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs" event={"ID":"fb8a1abb-7071-4712-82a9-b1ba1a96ba53","Type":"ContainerStarted","Data":"0791e822bbdb6e4c50467693e26918c490f3faee638d919b7f56e7b95d17fd30"}
Jul 15 20:06:19.039701 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:19.039312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs" event={"ID":"fb8a1abb-7071-4712-82a9-b1ba1a96ba53","Type":"ContainerStarted","Data":"c3e6e88f4dac4dbd7079e877f0aedccd47044adcada01b8cf821612c07221c10"}
Jul 15 20:06:19.060608 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:19.060552    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs" podStartSLOduration=1.8698366819999999 podStartE2EDuration="2.060536596s" totalImagesPullingTime="190.699914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:06:19.059420883 +0000 UTC m=+712.316823067" watchObservedRunningTime="2026-07-15 20:06:19.060536596 +0000 UTC m=+712.317938774"
Jul 15 20:06:27.897029 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:27.896991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:27.897499 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:27.897061    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:27.909636 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:27.909601    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:06:28.087704 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:06:28.087668    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs"
Jul 15 20:07:11.739862 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.739828    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56ddb87dd7-287jm"]
Jul 15 20:07:11.743115 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.743096    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:11.754007 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.753980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56ddb87dd7-287jm"]
Jul 15 20:07:11.808697 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.808660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxx8r\" (UniqueName: \"kubernetes.io/projected/09635fa1-9c2d-4147-9906-87835043fa1b-kube-api-access-vxx8r\") pod \"authorino-56ddb87dd7-287jm\" (UID: \"09635fa1-9c2d-4147-9906-87835043fa1b\") " pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:11.808894 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.808775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09635fa1-9c2d-4147-9906-87835043fa1b-tls-cert\") pod \"authorino-56ddb87dd7-287jm\" (UID: \"09635fa1-9c2d-4147-9906-87835043fa1b\") " pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:11.910257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.910211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxx8r\" (UniqueName: \"kubernetes.io/projected/09635fa1-9c2d-4147-9906-87835043fa1b-kube-api-access-vxx8r\") pod \"authorino-56ddb87dd7-287jm\" (UID: \"09635fa1-9c2d-4147-9906-87835043fa1b\") " pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:11.910438 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.910282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09635fa1-9c2d-4147-9906-87835043fa1b-tls-cert\") pod \"authorino-56ddb87dd7-287jm\" (UID: \"09635fa1-9c2d-4147-9906-87835043fa1b\") " pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:11.912963 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.912931    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09635fa1-9c2d-4147-9906-87835043fa1b-tls-cert\") pod \"authorino-56ddb87dd7-287jm\" (UID: \"09635fa1-9c2d-4147-9906-87835043fa1b\") " pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:11.919483 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:11.919458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxx8r\" (UniqueName: \"kubernetes.io/projected/09635fa1-9c2d-4147-9906-87835043fa1b-kube-api-access-vxx8r\") pod \"authorino-56ddb87dd7-287jm\" (UID: \"09635fa1-9c2d-4147-9906-87835043fa1b\") " pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:12.051653 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:12.051568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56ddb87dd7-287jm"
Jul 15 20:07:12.195157 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:12.195118    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56ddb87dd7-287jm"]
Jul 15 20:07:12.198349 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:07:12.198318    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09635fa1_9c2d_4147_9906_87835043fa1b.slice/crio-ab1cdacf01ccb87f922abf8e0d90e47cafd8c320d3ffd40c17e80955e38cd173 WatchSource:0}: Error finding container ab1cdacf01ccb87f922abf8e0d90e47cafd8c320d3ffd40c17e80955e38cd173: Status 404 returned error can't find the container with id ab1cdacf01ccb87f922abf8e0d90e47cafd8c320d3ffd40c17e80955e38cd173
Jul 15 20:07:12.256079 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:12.256040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56ddb87dd7-287jm" event={"ID":"09635fa1-9c2d-4147-9906-87835043fa1b","Type":"ContainerStarted","Data":"ab1cdacf01ccb87f922abf8e0d90e47cafd8c320d3ffd40c17e80955e38cd173"}
Jul 15 20:07:13.263548 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.263508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56ddb87dd7-287jm" event={"ID":"09635fa1-9c2d-4147-9906-87835043fa1b","Type":"ContainerStarted","Data":"9bd1abe8137e9f229bc9f683a5ebaa01704857843f28bb3c36cd51b5f288afb9"}
Jul 15 20:07:13.287058 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.286983    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56ddb87dd7-287jm" podStartSLOduration=1.8675140959999998 podStartE2EDuration="2.286956678s" totalImagesPullingTime="419.442582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:07:13.284899962 +0000 UTC m=+766.542302143" watchObservedRunningTime="2026-07-15 20:07:13.286956678 +0000 UTC m=+766.544358885"
Jul 15 20:07:13.324495 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.324451    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65d5dc4f7d-8qxlg"]
Jul 15 20:07:13.324771 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.324712    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg" podUID="639ddd84-354a-43bc-a0d6-a90b7d8e92c6" containerName="authorino" containerID="cri-o://4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50" gracePeriod=30
Jul 15 20:07:13.580150 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.580118    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:07:13.628832 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.628796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-tls-cert\") pod \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") "
Jul 15 20:07:13.628997 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.628855    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cnzd\" (UniqueName: \"kubernetes.io/projected/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-kube-api-access-9cnzd\") pod \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\" (UID: \"639ddd84-354a-43bc-a0d6-a90b7d8e92c6\") "
Jul 15 20:07:13.631095 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.631057    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-kube-api-access-9cnzd" (OuterVolumeSpecName: "kube-api-access-9cnzd") pod "639ddd84-354a-43bc-a0d6-a90b7d8e92c6" (UID: "639ddd84-354a-43bc-a0d6-a90b7d8e92c6"). InnerVolumeSpecName "kube-api-access-9cnzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:07:13.639131 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.639099    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "639ddd84-354a-43bc-a0d6-a90b7d8e92c6" (UID: "639ddd84-354a-43bc-a0d6-a90b7d8e92c6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:07:13.730274 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.730230    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-tls-cert\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:07:13.730274 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:13.730264    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cnzd\" (UniqueName: \"kubernetes.io/projected/639ddd84-354a-43bc-a0d6-a90b7d8e92c6-kube-api-access-9cnzd\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:07:14.269265 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.269176    2563 generic.go:356] "Generic (PLEG): container finished" podID="639ddd84-354a-43bc-a0d6-a90b7d8e92c6" containerID="4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50" exitCode=0
Jul 15 20:07:14.269265 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.269228    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg"
Jul 15 20:07:14.269745 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.269275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg" event={"ID":"639ddd84-354a-43bc-a0d6-a90b7d8e92c6","Type":"ContainerDied","Data":"4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50"}
Jul 15 20:07:14.269745 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.269328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d5dc4f7d-8qxlg" event={"ID":"639ddd84-354a-43bc-a0d6-a90b7d8e92c6","Type":"ContainerDied","Data":"2877940a72ef58293c9f3d9512c64664bd0a1c2d9a2761ede29fc862dab9cc37"}
Jul 15 20:07:14.269745 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.269353    2563 scope.go:117] "RemoveContainer" containerID="4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50"
Jul 15 20:07:14.279526 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.279502    2563 scope.go:117] "RemoveContainer" containerID="4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50"
Jul 15 20:07:14.279840 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:07:14.279820    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50\": container with ID starting with 4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50 not found: ID does not exist" containerID="4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50"
Jul 15 20:07:14.279929 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.279855    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50"} err="failed to get container status \"4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50\": rpc error: code = NotFound desc = could not find container \"4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50\": container with ID starting with 4310683f56585c70ef0dc46c1c7e4648f53975b46d92e24796750a33f303ae50 not found: ID does not exist"
Jul 15 20:07:14.316206 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.316167    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65d5dc4f7d-8qxlg"]
Jul 15 20:07:14.318368 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:14.318338    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65d5dc4f7d-8qxlg"]
Jul 15 20:07:15.298071 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:07:15.298036    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="639ddd84-354a-43bc-a0d6-a90b7d8e92c6" path="/var/lib/kubelet/pods/639ddd84-354a-43bc-a0d6-a90b7d8e92c6/volumes"
Jul 15 20:09:27.164131 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:09:27.164049    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:09:27.164131 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:09:27.164091    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:09:27.172031 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:09:27.172007    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:09:27.172031 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:09:27.172030    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:14:27.188097 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:14:27.188061    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:14:27.195240 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:14:27.195208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:14:27.201487 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:14:27.201466    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:14:27.202432 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:14:27.202407    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:19:27.217550 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:19:27.217522    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:19:27.220061 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:19:27.219788    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:19:27.224295 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:19:27.224276    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:19:27.226302 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:19:27.226286    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:24:27.239558 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:27.239434    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:24:27.247465 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:27.241958    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:24:27.247465 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:27.246208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:24:27.248529 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:27.248512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:24:55.347599 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.347559    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"]
Jul 15 20:24:55.348157 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.347977    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639ddd84-354a-43bc-a0d6-a90b7d8e92c6" containerName="authorino"
Jul 15 20:24:55.348157 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.347994    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="639ddd84-354a-43bc-a0d6-a90b7d8e92c6" containerName="authorino"
Jul 15 20:24:55.348157 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.348094    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="639ddd84-354a-43bc-a0d6-a90b7d8e92c6" containerName="authorino"
Jul 15 20:24:55.351051 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.351027    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"
Jul 15 20:24:55.355313 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.355292    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\""
Jul 15 20:24:55.355410 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.355329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:24:55.367283 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.367259    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"]
Jul 15 20:24:55.399710 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.399684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5126f6-aa30-4ac1-a741-5806fd9ca28a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-85451227-77lrg\" (UID: \"3f5126f6-aa30-4ac1-a741-5806fd9ca28a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"
Jul 15 20:24:55.500283 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.500258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5126f6-aa30-4ac1-a741-5806fd9ca28a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-85451227-77lrg\" (UID: \"3f5126f6-aa30-4ac1-a741-5806fd9ca28a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"
Jul 15 20:24:55.500831 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.500813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5126f6-aa30-4ac1-a741-5806fd9ca28a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-85451227-77lrg\" (UID: \"3f5126f6-aa30-4ac1-a741-5806fd9ca28a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"
Jul 15 20:24:55.660883 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.660860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"
Jul 15 20:24:55.791345 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.791323    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"]
Jul 15 20:24:55.793759 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:24:55.793704    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5126f6_aa30_4ac1_a741_5806fd9ca28a.slice/crio-0aaaf1e3ad441135c21572bef7bd1bdbe7b609195fc56784f5abf1d7372fa286 WatchSource:0}: Error finding container 0aaaf1e3ad441135c21572bef7bd1bdbe7b609195fc56784f5abf1d7372fa286: Status 404 returned error can't find the container with id 0aaaf1e3ad441135c21572bef7bd1bdbe7b609195fc56784f5abf1d7372fa286
Jul 15 20:24:55.795745 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:55.795704    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:24:56.655654 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:56.655619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg" event={"ID":"3f5126f6-aa30-4ac1-a741-5806fd9ca28a","Type":"ContainerStarted","Data":"0aaaf1e3ad441135c21572bef7bd1bdbe7b609195fc56784f5abf1d7372fa286"}
Jul 15 20:24:59.671188 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:59.671100    2563 generic.go:356] "Generic (PLEG): container finished" podID="3f5126f6-aa30-4ac1-a741-5806fd9ca28a" containerID="093b716d262e9347ac584379c8ec66764d05c4bea06fbff25f440128dac0ae71" exitCode=0
Jul 15 20:24:59.671188 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:24:59.671141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg" event={"ID":"3f5126f6-aa30-4ac1-a741-5806fd9ca28a","Type":"ContainerDied","Data":"093b716d262e9347ac584379c8ec66764d05c4bea06fbff25f440128dac0ae71"}
Jul 15 20:25:00.802226 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:25:00.802203    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"
Jul 15 20:25:00.842505 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:25:00.842479    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5126f6-aa30-4ac1-a741-5806fd9ca28a-maas-api-service-ca\") pod \"3f5126f6-aa30-4ac1-a741-5806fd9ca28a\" (UID: \"3f5126f6-aa30-4ac1-a741-5806fd9ca28a\") "
Jul 15 20:25:00.842863 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:25:00.842834    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5126f6-aa30-4ac1-a741-5806fd9ca28a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f5126f6-aa30-4ac1-a741-5806fd9ca28a" (UID: "3f5126f6-aa30-4ac1-a741-5806fd9ca28a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:25:00.943372 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:25:00.943307    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5126f6-aa30-4ac1-a741-5806fd9ca28a-maas-api-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:25:01.680368 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:25:01.680330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg" event={"ID":"3f5126f6-aa30-4ac1-a741-5806fd9ca28a","Type":"ContainerDied","Data":"0aaaf1e3ad441135c21572bef7bd1bdbe7b609195fc56784f5abf1d7372fa286"}
Jul 15 20:25:01.680544 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:25:01.680379    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0aaaf1e3ad441135c21572bef7bd1bdbe7b609195fc56784f5abf1d7372fa286"
Jul 15 20:25:01.680544 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:25:01.680416    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-77lrg"
Jul 15 20:27:14.845104 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.845073    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"]
Jul 15 20:27:14.845537 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.845347    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5126f6-aa30-4ac1-a741-5806fd9ca28a" containerName="revoke-keys"
Jul 15 20:27:14.845537 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.845358    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5126f6-aa30-4ac1-a741-5806fd9ca28a" containerName="revoke-keys"
Jul 15 20:27:14.845537 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.845419    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5126f6-aa30-4ac1-a741-5806fd9ca28a" containerName="revoke-keys"
Jul 15 20:27:14.848270 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.848252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"
Jul 15 20:27:14.850906 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.850881    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:27:14.852330 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.852303    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\""
Jul 15 20:27:14.860521 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.860491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"]
Jul 15 20:27:14.894983 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.894954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ecdea8d-ff96-4abf-855f-02254cf22b7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb\" (UID: \"9ecdea8d-ff96-4abf-855f-02254cf22b7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"
Jul 15 20:27:14.995996 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.995962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ecdea8d-ff96-4abf-855f-02254cf22b7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb\" (UID: \"9ecdea8d-ff96-4abf-855f-02254cf22b7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"
Jul 15 20:27:14.996576 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:14.996556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ecdea8d-ff96-4abf-855f-02254cf22b7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb\" (UID: \"9ecdea8d-ff96-4abf-855f-02254cf22b7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"
Jul 15 20:27:15.162393 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:15.162358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"
Jul 15 20:27:15.309226 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:15.309200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"]
Jul 15 20:27:15.311021 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:27:15.310994    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ecdea8d_ff96_4abf_855f_02254cf22b7a.slice/crio-d12c7150b3370ef7b1db16812954a57c7b9a9926edecd4faefaa4fc616cb66af WatchSource:0}: Error finding container d12c7150b3370ef7b1db16812954a57c7b9a9926edecd4faefaa4fc616cb66af: Status 404 returned error can't find the container with id d12c7150b3370ef7b1db16812954a57c7b9a9926edecd4faefaa4fc616cb66af
Jul 15 20:27:16.241050 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:16.240967    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ecdea8d-ff96-4abf-855f-02254cf22b7a" containerID="b3c2445160357b328ed6fba2fe64d8e11623cbeaa18cccc0ef7790d3a457ee25" exitCode=0
Jul 15 20:27:16.241404 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:16.241058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb" event={"ID":"9ecdea8d-ff96-4abf-855f-02254cf22b7a","Type":"ContainerDied","Data":"b3c2445160357b328ed6fba2fe64d8e11623cbeaa18cccc0ef7790d3a457ee25"}
Jul 15 20:27:16.241404 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:16.241118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb" event={"ID":"9ecdea8d-ff96-4abf-855f-02254cf22b7a","Type":"ContainerStarted","Data":"d12c7150b3370ef7b1db16812954a57c7b9a9926edecd4faefaa4fc616cb66af"}
Jul 15 20:27:17.366957 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:17.366934    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"
Jul 15 20:27:17.414644 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:17.414622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ecdea8d-ff96-4abf-855f-02254cf22b7a-maas-api-service-ca\") pod \"9ecdea8d-ff96-4abf-855f-02254cf22b7a\" (UID: \"9ecdea8d-ff96-4abf-855f-02254cf22b7a\") "
Jul 15 20:27:17.414990 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:17.414967    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ecdea8d-ff96-4abf-855f-02254cf22b7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ecdea8d-ff96-4abf-855f-02254cf22b7a" (UID: "9ecdea8d-ff96-4abf-855f-02254cf22b7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:27:17.516028 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:17.515961    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ecdea8d-ff96-4abf-855f-02254cf22b7a-maas-api-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:27:18.250875 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:18.250842    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb"
Jul 15 20:27:18.250875 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:18.250853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb" event={"ID":"9ecdea8d-ff96-4abf-855f-02254cf22b7a","Type":"ContainerDied","Data":"d12c7150b3370ef7b1db16812954a57c7b9a9926edecd4faefaa4fc616cb66af"}
Jul 15 20:27:18.251068 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:27:18.250899    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d12c7150b3370ef7b1db16812954a57c7b9a9926edecd4faefaa4fc616cb66af"
Jul 15 20:28:16.886129 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.886097    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"]
Jul 15 20:28:16.886676 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.886511    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ecdea8d-ff96-4abf-855f-02254cf22b7a" containerName="revoke-keys"
Jul 15 20:28:16.886676 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.886529    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ecdea8d-ff96-4abf-855f-02254cf22b7a" containerName="revoke-keys"
Jul 15 20:28:16.886676 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.886643    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ecdea8d-ff96-4abf-855f-02254cf22b7a" containerName="revoke-keys"
Jul 15 20:28:16.889648 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.889628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:16.893071 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.893045    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-83c1a2fd\"/\"e2e-lifecycle-model-83c1a2fd-kserve-self-signed-certs\""
Jul 15 20:28:16.894010 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.893993    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-83c1a2fd\"/\"kube-root-ca.crt\""
Jul 15 20:28:16.894098 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.894078    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-83c1a2fd\"/\"default-dockercfg-7vlp2\""
Jul 15 20:28:16.894178 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.894162    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-83c1a2fd\"/\"openshift-service-ca.crt\""
Jul 15 20:28:16.902044 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:16.902020    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"]
Jul 15 20:28:17.007091 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.007060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-dshm\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.007091 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.007094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qm9p\" (UniqueName: \"kubernetes.io/projected/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-kube-api-access-5qm9p\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.007251 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.007114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-home\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.007251 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.007154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tmp-dir\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.007251 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.007187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tls-certs\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.007349 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.007263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-model-cache\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108083 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-model-cache\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108232 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-dshm\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108232 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qm9p\" (UniqueName: \"kubernetes.io/projected/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-kube-api-access-5qm9p\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108232 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-home\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108405 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tmp-dir\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108405 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tls-certs\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108527 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-model-cache\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108579 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-home\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.108652 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.108634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tmp-dir\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.110608 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.110585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-dshm\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.110901 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.110882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tls-certs\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.117772 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.117750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qm9p\" (UniqueName: \"kubernetes.io/projected/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-kube-api-access-5qm9p\") pod \"e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") " pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.198836 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.198772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:17.339108 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.339077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"]
Jul 15 20:28:17.341626 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:28:17.341603    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod112c9657_9ae3_4a7b_a0b6_264d62a0dca2.slice/crio-ec1ff6193512d2fc77d8df6e0a22a799a918f2c1784b6b3d35b85f65e5117d89 WatchSource:0}: Error finding container ec1ff6193512d2fc77d8df6e0a22a799a918f2c1784b6b3d35b85f65e5117d89: Status 404 returned error can't find the container with id ec1ff6193512d2fc77d8df6e0a22a799a918f2c1784b6b3d35b85f65e5117d89
Jul 15 20:28:17.495548 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.495460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24" event={"ID":"112c9657-9ae3-4a7b-a0b6-264d62a0dca2","Type":"ContainerStarted","Data":"91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59"}
Jul 15 20:28:17.495548 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.495511    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24" event={"ID":"112c9657-9ae3-4a7b-a0b6-264d62a0dca2","Type":"ContainerStarted","Data":"ec1ff6193512d2fc77d8df6e0a22a799a918f2c1784b6b3d35b85f65e5117d89"}
Jul 15 20:28:17.517006 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:17.516955    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24" podStartSLOduration=1.516939164 podStartE2EDuration="1.516939164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:28:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:28:17.515986089 +0000 UTC m=+2030.773388269" watchObservedRunningTime="2026-07-15 20:28:17.516939164 +0000 UTC m=+2030.774341365"
Jul 15 20:28:27.199109 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:27.199068    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:27.199671 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:27.199324    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:27.212085 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:27.212059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:28:27.547816 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:28:27.547706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:29:02.450081 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:02.449998    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"]
Jul 15 20:29:17.470310 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.470273    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24" podUID="112c9657-9ae3-4a7b-a0b6-264d62a0dca2" containerName="main" containerID="cri-o://91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59" gracePeriod=45
Jul 15 20:29:17.536555 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.536524    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24" podUID="112c9657-9ae3-4a7b-a0b6-264d62a0dca2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": dial tcp 10.134.0.33:8000: connect: connection refused"
Jul 15 20:29:17.715357 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.715335    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:29:17.744171 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.744097    2563 generic.go:356] "Generic (PLEG): container finished" podID="112c9657-9ae3-4a7b-a0b6-264d62a0dca2" containerID="91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59" exitCode=0
Jul 15 20:29:17.744307 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.744191    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"
Jul 15 20:29:17.744307 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.744229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24" event={"ID":"112c9657-9ae3-4a7b-a0b6-264d62a0dca2","Type":"ContainerDied","Data":"91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59"}
Jul 15 20:29:17.744307 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.744273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24" event={"ID":"112c9657-9ae3-4a7b-a0b6-264d62a0dca2","Type":"ContainerDied","Data":"ec1ff6193512d2fc77d8df6e0a22a799a918f2c1784b6b3d35b85f65e5117d89"}
Jul 15 20:29:17.744307 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.744298    2563 scope.go:117] "RemoveContainer" containerID="91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59"
Jul 15 20:29:17.754147 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.754080    2563 scope.go:117] "RemoveContainer" containerID="91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59"
Jul 15 20:29:17.754379 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:29:17.754357    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59\": container with ID starting with 91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59 not found: ID does not exist" containerID="91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59"
Jul 15 20:29:17.754446 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.754384    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59"} err="failed to get container status \"91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59\": rpc error: code = NotFound desc = could not find container \"91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59\": container with ID starting with 91767ab5d56ee80178d7088e731451dfe684015467c63d1c95093cdc34b6de59 not found: ID does not exist"
Jul 15 20:29:17.808793 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.808766    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qm9p\" (UniqueName: \"kubernetes.io/projected/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-kube-api-access-5qm9p\") pod \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") "
Jul 15 20:29:17.808913 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.808813    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tmp-dir\") pod \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") "
Jul 15 20:29:17.808913 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.808833    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tls-certs\") pod \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") "
Jul 15 20:29:17.808913 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.808870    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-dshm\") pod \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") "
Jul 15 20:29:17.808913 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.808893    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-model-cache\") pod \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") "
Jul 15 20:29:17.809130 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.808928    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-home\") pod \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\" (UID: \"112c9657-9ae3-4a7b-a0b6-264d62a0dca2\") "
Jul 15 20:29:17.809130 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.809068    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "112c9657-9ae3-4a7b-a0b6-264d62a0dca2" (UID: "112c9657-9ae3-4a7b-a0b6-264d62a0dca2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:29:17.809239 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.809147    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-model-cache" (OuterVolumeSpecName: "model-cache") pod "112c9657-9ae3-4a7b-a0b6-264d62a0dca2" (UID: "112c9657-9ae3-4a7b-a0b6-264d62a0dca2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:29:17.809239 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.809200    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:29:17.809239 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.809218    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:29:17.809356 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.809272    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-home" (OuterVolumeSpecName: "home") pod "112c9657-9ae3-4a7b-a0b6-264d62a0dca2" (UID: "112c9657-9ae3-4a7b-a0b6-264d62a0dca2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:29:17.811067 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.811044    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-dshm" (OuterVolumeSpecName: "dshm") pod "112c9657-9ae3-4a7b-a0b6-264d62a0dca2" (UID: "112c9657-9ae3-4a7b-a0b6-264d62a0dca2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:29:17.811161 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.811065    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-kube-api-access-5qm9p" (OuterVolumeSpecName: "kube-api-access-5qm9p") pod "112c9657-9ae3-4a7b-a0b6-264d62a0dca2" (UID: "112c9657-9ae3-4a7b-a0b6-264d62a0dca2"). InnerVolumeSpecName "kube-api-access-5qm9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:29:17.811161 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.811095    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "112c9657-9ae3-4a7b-a0b6-264d62a0dca2" (UID: "112c9657-9ae3-4a7b-a0b6-264d62a0dca2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:29:17.909839 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.909817    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:29:17.909839 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.909840    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:29:17.909971 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.909848    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:29:17.909971 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:17.909856    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qm9p\" (UniqueName: \"kubernetes.io/projected/112c9657-9ae3-4a7b-a0b6-264d62a0dca2-kube-api-access-5qm9p\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:29:18.068425 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:18.068401    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"]
Jul 15 20:29:18.070392 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:18.070372    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-83c1a2fd/e2e-lifecycle-model-83c1a2fd-kserve-56c69b6b7f-77s24"]
Jul 15 20:29:19.296791 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:19.296756    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="112c9657-9ae3-4a7b-a0b6-264d62a0dca2" path="/var/lib/kubelet/pods/112c9657-9ae3-4a7b-a0b6-264d62a0dca2/volumes"
Jul 15 20:29:27.265495 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:27.265386    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:29:27.280406 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:27.269276    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:29:27.280406 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:27.272636    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:29:27.280406 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:29:27.276296    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:31:04.542257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.542228    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"]
Jul 15 20:31:04.542691 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.542507    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="112c9657-9ae3-4a7b-a0b6-264d62a0dca2" containerName="main"
Jul 15 20:31:04.542691 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.542518    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="112c9657-9ae3-4a7b-a0b6-264d62a0dca2" containerName="main"
Jul 15 20:31:04.542691 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.542569    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="112c9657-9ae3-4a7b-a0b6-264d62a0dca2" containerName="main"
Jul 15 20:31:04.545535 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.545513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.550358 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.550336    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bec571\"/\"default-dockercfg-kb4lh\""
Jul 15 20:31:04.551289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.551268    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bec571\"/\"kube-root-ca.crt\""
Jul 15 20:31:04.551394 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.551291    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bec571\"/\"openshift-service-ca.crt\""
Jul 15 20:31:04.551892 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.551693    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bec571\"/\"test-model-bec571-kserve-self-signed-certs\""
Jul 15 20:31:04.585998 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.585967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kscgc\" (UniqueName: \"kubernetes.io/projected/f40a64c3-fc3d-472b-bd13-52e0a045f98a-kube-api-access-kscgc\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.585998 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.585999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-model-cache\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.586162 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.586025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tls-certs\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.586162 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.586106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-dshm\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.586162 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.586153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tmp-dir\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.586289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.586211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-home\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.592459 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.592430    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"]
Jul 15 20:31:04.687071 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-home\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687253 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kscgc\" (UniqueName: \"kubernetes.io/projected/f40a64c3-fc3d-472b-bd13-52e0a045f98a-kube-api-access-kscgc\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687253 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-model-cache\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687253 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tls-certs\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687253 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-dshm\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687253 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tmp-dir\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687508 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-home\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687508 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-model-cache\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.687609 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.687551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tmp-dir\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.689584 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.689564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-dshm\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.689879 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.689860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tls-certs\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.696925 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.696907    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kscgc\" (UniqueName: \"kubernetes.io/projected/f40a64c3-fc3d-472b-bd13-52e0a045f98a-kube-api-access-kscgc\") pod \"test-model-bec571-kserve-65f5cd8bdc-9529l\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") " pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.854251 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.854224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:04.993689 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:04.993667    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"]
Jul 15 20:31:04.996405 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:31:04.996380    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf40a64c3_fc3d_472b_bd13_52e0a045f98a.slice/crio-bec65cea30c27975abb0bc4eb55caba4f8550b538c7e9ab73ac61a06d286c176 WatchSource:0}: Error finding container bec65cea30c27975abb0bc4eb55caba4f8550b538c7e9ab73ac61a06d286c176: Status 404 returned error can't find the container with id bec65cea30c27975abb0bc4eb55caba4f8550b538c7e9ab73ac61a06d286c176
Jul 15 20:31:05.190407 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:05.190318    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l" event={"ID":"f40a64c3-fc3d-472b-bd13-52e0a045f98a","Type":"ContainerStarted","Data":"78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63"}
Jul 15 20:31:05.190407 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:05.190362    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l" event={"ID":"f40a64c3-fc3d-472b-bd13-52e0a045f98a","Type":"ContainerStarted","Data":"bec65cea30c27975abb0bc4eb55caba4f8550b538c7e9ab73ac61a06d286c176"}
Jul 15 20:31:05.211293 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:05.211238    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l" podStartSLOduration=1.211219576 podStartE2EDuration="1.211219576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:31:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:31:05.210410492 +0000 UTC m=+2198.467812670" watchObservedRunningTime="2026-07-15 20:31:05.211219576 +0000 UTC m=+2198.468621755"
Jul 15 20:31:14.854973 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:14.854920    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:14.854973 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:14.854985    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:14.867076 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:14.867049    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:15.246929 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:15.246842    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:31:32.213445 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.213411    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"]
Jul 15 20:31:32.216895 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.216869    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.222622 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.222514    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4d97ce\"/\"openshift-service-ca.crt\""
Jul 15 20:31:32.223423 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.223397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4d97ce\"/\"kube-root-ca.crt\""
Jul 15 20:31:32.223544 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.223404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4d97ce\"/\"default-dockercfg-wrsq5\""
Jul 15 20:31:32.223605 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.223563    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4d97ce\"/\"test-model-4d97ce-kserve-self-signed-certs\""
Jul 15 20:31:32.242369 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.242344    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"]
Jul 15 20:31:32.313870 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.313829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-home\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.314048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.313912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65349f41-4d52-4947-bace-d3b47d38c71b-tls-certs\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.314048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.313938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jljr\" (UniqueName: \"kubernetes.io/projected/65349f41-4d52-4947-bace-d3b47d38c71b-kube-api-access-9jljr\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.314048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.313987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-dshm\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.314048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.314010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-model-cache\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.314183 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.314107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-tmp-dir\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.415431 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.415388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65349f41-4d52-4947-bace-d3b47d38c71b-tls-certs\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.415431 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.415430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jljr\" (UniqueName: \"kubernetes.io/projected/65349f41-4d52-4947-bace-d3b47d38c71b-kube-api-access-9jljr\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.415682 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.415452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-dshm\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.415682 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.415469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-model-cache\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.415682 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.415537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-tmp-dir\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.415682 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.415674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-home\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.416197 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.416172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-tmp-dir\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.416318 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.416204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-model-cache\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.416391 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.416250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-home\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.418444 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.418421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-dshm\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.418838 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.418817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65349f41-4d52-4947-bace-d3b47d38c71b-tls-certs\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.428477 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.428443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jljr\" (UniqueName: \"kubernetes.io/projected/65349f41-4d52-4947-bace-d3b47d38c71b-kube-api-access-9jljr\") pod \"test-model-4d97ce-kserve-7b487c4457-4s6t7\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") " pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.529677 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.529601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:32.698512 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:32.698479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"]
Jul 15 20:31:32.701761 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:31:32.701737    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65349f41_4d52_4947_bace_d3b47d38c71b.slice/crio-a4a3fdf9b2a868aca680b2a4bb15242b3e26476007b68915fab251d5ccf59f75 WatchSource:0}: Error finding container a4a3fdf9b2a868aca680b2a4bb15242b3e26476007b68915fab251d5ccf59f75: Status 404 returned error can't find the container with id a4a3fdf9b2a868aca680b2a4bb15242b3e26476007b68915fab251d5ccf59f75
Jul 15 20:31:33.311137 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:33.311096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7" event={"ID":"65349f41-4d52-4947-bace-d3b47d38c71b","Type":"ContainerStarted","Data":"777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77"}
Jul 15 20:31:33.311576 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:33.311145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7" event={"ID":"65349f41-4d52-4947-bace-d3b47d38c71b","Type":"ContainerStarted","Data":"a4a3fdf9b2a868aca680b2a4bb15242b3e26476007b68915fab251d5ccf59f75"}
Jul 15 20:31:33.341644 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:33.341579    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7" podStartSLOduration=1.341557774 podStartE2EDuration="1.341557774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:31:33.340385808 +0000 UTC m=+2226.597787999" watchObservedRunningTime="2026-07-15 20:31:33.341557774 +0000 UTC m=+2226.598960053"
Jul 15 20:31:42.529914 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:42.529873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:42.530529 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:42.529926    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:42.542321 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:42.542296    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:31:43.363892 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:31:43.363864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:32:06.166563 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:06.166524    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"]
Jul 15 20:32:07.095453 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:07.095416    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"]
Jul 15 20:32:21.186759 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.186697    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l" podUID="f40a64c3-fc3d-472b-bd13-52e0a045f98a" containerName="main" containerID="cri-o://78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63" gracePeriod=45
Jul 15 20:32:21.430185 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.430159    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:32:21.512683 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.512594    2563 generic.go:356] "Generic (PLEG): container finished" podID="f40a64c3-fc3d-472b-bd13-52e0a045f98a" containerID="78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63" exitCode=0
Jul 15 20:32:21.512683 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.512637    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"
Jul 15 20:32:21.512683 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.512646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l" event={"ID":"f40a64c3-fc3d-472b-bd13-52e0a045f98a","Type":"ContainerDied","Data":"78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63"}
Jul 15 20:32:21.512683 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.512684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l" event={"ID":"f40a64c3-fc3d-472b-bd13-52e0a045f98a","Type":"ContainerDied","Data":"bec65cea30c27975abb0bc4eb55caba4f8550b538c7e9ab73ac61a06d286c176"}
Jul 15 20:32:21.512976 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.512702    2563 scope.go:117] "RemoveContainer" containerID="78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63"
Jul 15 20:32:21.521257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.521234    2563 scope.go:117] "RemoveContainer" containerID="78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63"
Jul 15 20:32:21.521523 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:32:21.521506    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63\": container with ID starting with 78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63 not found: ID does not exist" containerID="78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63"
Jul 15 20:32:21.521579 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.521538    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63"} err="failed to get container status \"78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63\": rpc error: code = NotFound desc = could not find container \"78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63\": container with ID starting with 78bdb23a5f7f23e442269cb747a2d242d76d7f6bcb5275c45fc3327b0ed5ca63 not found: ID does not exist"
Jul 15 20:32:21.565516 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565493    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kscgc\" (UniqueName: \"kubernetes.io/projected/f40a64c3-fc3d-472b-bd13-52e0a045f98a-kube-api-access-kscgc\") pod \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") "
Jul 15 20:32:21.565618 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565555    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tmp-dir\") pod \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") "
Jul 15 20:32:21.565618 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-model-cache\") pod \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") "
Jul 15 20:32:21.565618 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-dshm\") pod \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") "
Jul 15 20:32:21.565739 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565670    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tls-certs\") pod \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") "
Jul 15 20:32:21.565739 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565705    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-home\") pod \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\" (UID: \"f40a64c3-fc3d-472b-bd13-52e0a045f98a\") "
Jul 15 20:32:21.565898 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-model-cache" (OuterVolumeSpecName: "model-cache") pod "f40a64c3-fc3d-472b-bd13-52e0a045f98a" (UID: "f40a64c3-fc3d-472b-bd13-52e0a045f98a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:21.565898 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.565880    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f40a64c3-fc3d-472b-bd13-52e0a045f98a" (UID: "f40a64c3-fc3d-472b-bd13-52e0a045f98a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:21.566073 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.566046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-home" (OuterVolumeSpecName: "home") pod "f40a64c3-fc3d-472b-bd13-52e0a045f98a" (UID: "f40a64c3-fc3d-472b-bd13-52e0a045f98a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:21.566161 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.566070    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:21.566161 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.566090    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:21.567867 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.567844    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-dshm" (OuterVolumeSpecName: "dshm") pod "f40a64c3-fc3d-472b-bd13-52e0a045f98a" (UID: "f40a64c3-fc3d-472b-bd13-52e0a045f98a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:21.567943 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.567909    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f40a64c3-fc3d-472b-bd13-52e0a045f98a" (UID: "f40a64c3-fc3d-472b-bd13-52e0a045f98a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:32:21.567943 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.567931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f40a64c3-fc3d-472b-bd13-52e0a045f98a-kube-api-access-kscgc" (OuterVolumeSpecName: "kube-api-access-kscgc") pod "f40a64c3-fc3d-472b-bd13-52e0a045f98a" (UID: "f40a64c3-fc3d-472b-bd13-52e0a045f98a"). InnerVolumeSpecName "kube-api-access-kscgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:32:21.666556 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.666521    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:21.666556 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.666553    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40a64c3-fc3d-472b-bd13-52e0a045f98a-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:21.666556 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.666564    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40a64c3-fc3d-472b-bd13-52e0a045f98a-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:21.666830 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.666573    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kscgc\" (UniqueName: \"kubernetes.io/projected/f40a64c3-fc3d-472b-bd13-52e0a045f98a-kube-api-access-kscgc\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:21.850566 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.850533    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"]
Jul 15 20:32:21.859351 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:21.859327    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-bec571/test-model-bec571-kserve-65f5cd8bdc-9529l"]
Jul 15 20:32:22.115729 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.115690    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7" podUID="65349f41-4d52-4947-bace-d3b47d38c71b" containerName="main" containerID="cri-o://777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77" gracePeriod=45
Jul 15 20:32:22.357462 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.357439    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:32:22.472577 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472488    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-tmp-dir\") pod \"65349f41-4d52-4947-bace-d3b47d38c71b\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") "
Jul 15 20:32:22.472577 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472542    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jljr\" (UniqueName: \"kubernetes.io/projected/65349f41-4d52-4947-bace-d3b47d38c71b-kube-api-access-9jljr\") pod \"65349f41-4d52-4947-bace-d3b47d38c71b\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") "
Jul 15 20:32:22.472577 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472570    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65349f41-4d52-4947-bace-d3b47d38c71b-tls-certs\") pod \"65349f41-4d52-4947-bace-d3b47d38c71b\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") "
Jul 15 20:32:22.472818 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-dshm\") pod \"65349f41-4d52-4947-bace-d3b47d38c71b\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") "
Jul 15 20:32:22.472818 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472628    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-model-cache\") pod \"65349f41-4d52-4947-bace-d3b47d38c71b\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") "
Jul 15 20:32:22.472885 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472812    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-home\") pod \"65349f41-4d52-4947-bace-d3b47d38c71b\" (UID: \"65349f41-4d52-4947-bace-d3b47d38c71b\") "
Jul 15 20:32:22.472885 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65349f41-4d52-4947-bace-d3b47d38c71b" (UID: "65349f41-4d52-4947-bace-d3b47d38c71b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:22.472953 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.472928    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-model-cache" (OuterVolumeSpecName: "model-cache") pod "65349f41-4d52-4947-bace-d3b47d38c71b" (UID: "65349f41-4d52-4947-bace-d3b47d38c71b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:22.473093 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.473070    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-home" (OuterVolumeSpecName: "home") pod "65349f41-4d52-4947-bace-d3b47d38c71b" (UID: "65349f41-4d52-4947-bace-d3b47d38c71b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:22.473203 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.473136    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:22.473203 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.473150    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:22.473203 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.473162    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:22.475013 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.474985    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65349f41-4d52-4947-bace-d3b47d38c71b-kube-api-access-9jljr" (OuterVolumeSpecName: "kube-api-access-9jljr") pod "65349f41-4d52-4947-bace-d3b47d38c71b" (UID: "65349f41-4d52-4947-bace-d3b47d38c71b"). InnerVolumeSpecName "kube-api-access-9jljr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:32:22.475274 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.475244    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65349f41-4d52-4947-bace-d3b47d38c71b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65349f41-4d52-4947-bace-d3b47d38c71b" (UID: "65349f41-4d52-4947-bace-d3b47d38c71b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:32:22.475352 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.475244    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-dshm" (OuterVolumeSpecName: "dshm") pod "65349f41-4d52-4947-bace-d3b47d38c71b" (UID: "65349f41-4d52-4947-bace-d3b47d38c71b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:32:22.518007 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.517978    2563 generic.go:356] "Generic (PLEG): container finished" podID="65349f41-4d52-4947-bace-d3b47d38c71b" containerID="777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77" exitCode=0
Jul 15 20:32:22.518170 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.518022    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"
Jul 15 20:32:22.518170 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.518066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7" event={"ID":"65349f41-4d52-4947-bace-d3b47d38c71b","Type":"ContainerDied","Data":"777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77"}
Jul 15 20:32:22.518170 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.518104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7" event={"ID":"65349f41-4d52-4947-bace-d3b47d38c71b","Type":"ContainerDied","Data":"a4a3fdf9b2a868aca680b2a4bb15242b3e26476007b68915fab251d5ccf59f75"}
Jul 15 20:32:22.518170 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.518127    2563 scope.go:117] "RemoveContainer" containerID="777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77"
Jul 15 20:32:22.528949 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.528922    2563 scope.go:117] "RemoveContainer" containerID="777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77"
Jul 15 20:32:22.529432 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:32:22.529408    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77\": container with ID starting with 777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77 not found: ID does not exist" containerID="777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77"
Jul 15 20:32:22.529508 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.529444    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77"} err="failed to get container status \"777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77\": rpc error: code = NotFound desc = could not find container \"777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77\": container with ID starting with 777daf3136da9ac228da495c9136fcb124d21e28f3807f7bcac6c4fd4ddf1f77 not found: ID does not exist"
Jul 15 20:32:22.543499 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.543467    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"]
Jul 15 20:32:22.551289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.551268    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4d97ce/test-model-4d97ce-kserve-7b487c4457-4s6t7"]
Jul 15 20:32:22.574410 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.574376    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65349f41-4d52-4947-bace-d3b47d38c71b-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:22.574410 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.574406    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jljr\" (UniqueName: \"kubernetes.io/projected/65349f41-4d52-4947-bace-d3b47d38c71b-kube-api-access-9jljr\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:22.574569 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:22.574421    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65349f41-4d52-4947-bace-d3b47d38c71b-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:23.296955 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:23.296923    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65349f41-4d52-4947-bace-d3b47d38c71b" path="/var/lib/kubelet/pods/65349f41-4d52-4947-bace-d3b47d38c71b/volumes"
Jul 15 20:32:23.297329 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:23.297314    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f40a64c3-fc3d-472b-bd13-52e0a045f98a" path="/var/lib/kubelet/pods/f40a64c3-fc3d-472b-bd13-52e0a045f98a/volumes"
Jul 15 20:32:33.545386 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.545345    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"]
Jul 15 20:32:33.545797 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.545629    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f40a64c3-fc3d-472b-bd13-52e0a045f98a" containerName="main"
Jul 15 20:32:33.545797 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.545639    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f40a64c3-fc3d-472b-bd13-52e0a045f98a" containerName="main"
Jul 15 20:32:33.545797 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.545652    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65349f41-4d52-4947-bace-d3b47d38c71b" containerName="main"
Jul 15 20:32:33.545797 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.545657    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="65349f41-4d52-4947-bace-d3b47d38c71b" containerName="main"
Jul 15 20:32:33.545797 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.545706    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f40a64c3-fc3d-472b-bd13-52e0a045f98a" containerName="main"
Jul 15 20:32:33.545797 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.545712    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="65349f41-4d52-4947-bace-d3b47d38c71b" containerName="main"
Jul 15 20:32:33.548661 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.548643    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"
Jul 15 20:32:33.551784 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.551759    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:32:33.552126 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.551762    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\""
Jul 15 20:32:33.560902 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.560878    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"]
Jul 15 20:32:33.660117 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.660073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx\" (UID: \"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"
Jul 15 20:32:33.761181 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.761138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx\" (UID: \"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"
Jul 15 20:32:33.761814 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.761794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx\" (UID: \"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"
Jul 15 20:32:33.860856 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:33.860823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"
Jul 15 20:32:34.002000 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:34.001964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"]
Jul 15 20:32:34.004756 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:32:34.004694    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2e9dbc_0fa9_4eb7_bf42_b29e8d2d5233.slice/crio-5f9967279190afff16c52724b4ce8e4a0fc2acbc6d08fe609b176b166a3869af WatchSource:0}: Error finding container 5f9967279190afff16c52724b4ce8e4a0fc2acbc6d08fe609b176b166a3869af: Status 404 returned error can't find the container with id 5f9967279190afff16c52724b4ce8e4a0fc2acbc6d08fe609b176b166a3869af
Jul 15 20:32:34.572286 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:34.572202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx" event={"ID":"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233","Type":"ContainerStarted","Data":"a20e1730114ae7cacca4f138c54d5abf080cc70935817976d47cc337ce597f90"}
Jul 15 20:32:34.572286 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:34.572257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx" event={"ID":"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233","Type":"ContainerStarted","Data":"5f9967279190afff16c52724b4ce8e4a0fc2acbc6d08fe609b176b166a3869af"}
Jul 15 20:32:34.594584 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:34.594532    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx" podStartSLOduration=1.594515646 podStartE2EDuration="1.594515646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:32:34.593554817 +0000 UTC m=+2287.850956998" watchObservedRunningTime="2026-07-15 20:32:34.594515646 +0000 UTC m=+2287.851917824"
Jul 15 20:32:35.577668 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:35.577631    2563 generic.go:356] "Generic (PLEG): container finished" podID="7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233" containerID="a20e1730114ae7cacca4f138c54d5abf080cc70935817976d47cc337ce597f90" exitCode=0
Jul 15 20:32:35.578056 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:35.577712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx" event={"ID":"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233","Type":"ContainerDied","Data":"a20e1730114ae7cacca4f138c54d5abf080cc70935817976d47cc337ce597f90"}
Jul 15 20:32:36.712398 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:36.712373    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"
Jul 15 20:32:36.785745 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:36.785673    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233-maas-api-service-ca\") pod \"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233\" (UID: \"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233\") "
Jul 15 20:32:36.786031 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:36.786007    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233" (UID: "7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:32:36.887052 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:36.887021    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233-maas-api-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:32:37.588415 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:37.588338    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx"
Jul 15 20:32:37.588415 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:37.588346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx" event={"ID":"7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233","Type":"ContainerDied","Data":"5f9967279190afff16c52724b4ce8e4a0fc2acbc6d08fe609b176b166a3869af"}
Jul 15 20:32:37.588415 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:32:37.588393    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f9967279190afff16c52724b4ce8e4a0fc2acbc6d08fe609b176b166a3869af"
Jul 15 20:33:25.628698 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.628616    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"]
Jul 15 20:33:25.629133 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.628921    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233" containerName="revoke-keys"
Jul 15 20:33:25.629133 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.628934    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233" containerName="revoke-keys"
Jul 15 20:33:25.629133 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.628992    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233" containerName="revoke-keys"
Jul 15 20:33:25.631980 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.631964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"
Jul 15 20:33:25.637902 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.637882    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\""
Jul 15 20:33:25.638520 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.638496    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:33:25.646550 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.646526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"]
Jul 15 20:33:25.705966 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.705927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0438f1d-56f4-4e4a-aae1-df01352a8f90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-514f9c-56x2k\" (UID: \"f0438f1d-56f4-4e4a-aae1-df01352a8f90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"
Jul 15 20:33:25.806464 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.806427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0438f1d-56f4-4e4a-aae1-df01352a8f90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-514f9c-56x2k\" (UID: \"f0438f1d-56f4-4e4a-aae1-df01352a8f90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"
Jul 15 20:33:25.807117 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.807090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0438f1d-56f4-4e4a-aae1-df01352a8f90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-514f9c-56x2k\" (UID: \"f0438f1d-56f4-4e4a-aae1-df01352a8f90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"
Jul 15 20:33:25.942178 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:25.942090    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"
Jul 15 20:33:26.102190 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:26.102156    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"]
Jul 15 20:33:26.104429 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:33:26.104402    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0438f1d_56f4_4e4a_aae1_df01352a8f90.slice/crio-e9f82a44e500c5d97550c2f73fd0f10178f1ba4e17081e1e6060679bad6138e6 WatchSource:0}: Error finding container e9f82a44e500c5d97550c2f73fd0f10178f1ba4e17081e1e6060679bad6138e6: Status 404 returned error can't find the container with id e9f82a44e500c5d97550c2f73fd0f10178f1ba4e17081e1e6060679bad6138e6
Jul 15 20:33:26.807482 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:26.807386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k" event={"ID":"f0438f1d-56f4-4e4a-aae1-df01352a8f90","Type":"ContainerStarted","Data":"bc98f43ca327b5a9b8a9cc1f6510e49b31dc0ed629f1bac281ce6301cf008bf7"}
Jul 15 20:33:26.807482 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:26.807442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k" event={"ID":"f0438f1d-56f4-4e4a-aae1-df01352a8f90","Type":"ContainerStarted","Data":"e9f82a44e500c5d97550c2f73fd0f10178f1ba4e17081e1e6060679bad6138e6"}
Jul 15 20:33:27.812870 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:27.812829    2563 generic.go:356] "Generic (PLEG): container finished" podID="f0438f1d-56f4-4e4a-aae1-df01352a8f90" containerID="bc98f43ca327b5a9b8a9cc1f6510e49b31dc0ed629f1bac281ce6301cf008bf7" exitCode=0
Jul 15 20:33:27.812870 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:27.812865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k" event={"ID":"f0438f1d-56f4-4e4a-aae1-df01352a8f90","Type":"ContainerDied","Data":"bc98f43ca327b5a9b8a9cc1f6510e49b31dc0ed629f1bac281ce6301cf008bf7"}
Jul 15 20:33:28.949275 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:28.949250    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"
Jul 15 20:33:29.033821 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:29.033779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0438f1d-56f4-4e4a-aae1-df01352a8f90-maas-api-service-ca\") pod \"f0438f1d-56f4-4e4a-aae1-df01352a8f90\" (UID: \"f0438f1d-56f4-4e4a-aae1-df01352a8f90\") "
Jul 15 20:33:29.034146 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:29.034121    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0438f1d-56f4-4e4a-aae1-df01352a8f90-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f0438f1d-56f4-4e4a-aae1-df01352a8f90" (UID: "f0438f1d-56f4-4e4a-aae1-df01352a8f90"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:33:29.134502 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:29.134470    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0438f1d-56f4-4e4a-aae1-df01352a8f90-maas-api-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:33:29.822075 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:29.822045    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k"
Jul 15 20:33:29.822075 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:29.822051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-514f9c-56x2k" event={"ID":"f0438f1d-56f4-4e4a-aae1-df01352a8f90","Type":"ContainerDied","Data":"e9f82a44e500c5d97550c2f73fd0f10178f1ba4e17081e1e6060679bad6138e6"}
Jul 15 20:33:29.822276 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:29.822095    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9f82a44e500c5d97550c2f73fd0f10178f1ba4e17081e1e6060679bad6138e6"
Jul 15 20:33:53.324860 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.324824    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"]
Jul 15 20:33:53.325246 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.325090    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0438f1d-56f4-4e4a-aae1-df01352a8f90" containerName="revoke-keys"
Jul 15 20:33:53.325246 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.325101    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0438f1d-56f4-4e4a-aae1-df01352a8f90" containerName="revoke-keys"
Jul 15 20:33:53.325246 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.325164    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0438f1d-56f4-4e4a-aae1-df01352a8f90" containerName="revoke-keys"
Jul 15 20:33:53.327935 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.327919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"
Jul 15 20:33:53.330452 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.330426    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\""
Jul 15 20:33:53.330566 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.330463    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:33:53.341471 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.341446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"]
Jul 15 20:33:53.433453 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.433416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7606859c-2d51-4d22-8cbb-c0701c922c3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h\" (UID: \"7606859c-2d51-4d22-8cbb-c0701c922c3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"
Jul 15 20:33:53.534435 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.534404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7606859c-2d51-4d22-8cbb-c0701c922c3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h\" (UID: \"7606859c-2d51-4d22-8cbb-c0701c922c3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"
Jul 15 20:33:53.535074 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.535050    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7606859c-2d51-4d22-8cbb-c0701c922c3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h\" (UID: \"7606859c-2d51-4d22-8cbb-c0701c922c3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"
Jul 15 20:33:53.638489 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.638456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"
Jul 15 20:33:53.768326 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.768178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"]
Jul 15 20:33:53.771000 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:33:53.770973    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7606859c_2d51_4d22_8cbb_c0701c922c3b.slice/crio-b4c16d2de292c3d405e6504d65c7094791b0d737662ae89196186f516700eb96 WatchSource:0}: Error finding container b4c16d2de292c3d405e6504d65c7094791b0d737662ae89196186f516700eb96: Status 404 returned error can't find the container with id b4c16d2de292c3d405e6504d65c7094791b0d737662ae89196186f516700eb96
Jul 15 20:33:53.935649 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:53.935570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h" event={"ID":"7606859c-2d51-4d22-8cbb-c0701c922c3b","Type":"ContainerStarted","Data":"b4c16d2de292c3d405e6504d65c7094791b0d737662ae89196186f516700eb96"}
Jul 15 20:33:54.940841 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:54.940807    2563 generic.go:356] "Generic (PLEG): container finished" podID="7606859c-2d51-4d22-8cbb-c0701c922c3b" containerID="e91ca7e05b140bde150821142f44c7ad49babcbbe098047048340152eeb6c528" exitCode=0
Jul 15 20:33:54.941211 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:54.940858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h" event={"ID":"7606859c-2d51-4d22-8cbb-c0701c922c3b","Type":"ContainerDied","Data":"e91ca7e05b140bde150821142f44c7ad49babcbbe098047048340152eeb6c528"}
Jul 15 20:33:56.073141 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:56.073110    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"
Jul 15 20:33:56.156047 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:56.156018    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7606859c-2d51-4d22-8cbb-c0701c922c3b-maas-api-service-ca\") pod \"7606859c-2d51-4d22-8cbb-c0701c922c3b\" (UID: \"7606859c-2d51-4d22-8cbb-c0701c922c3b\") "
Jul 15 20:33:56.156370 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:56.156349    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7606859c-2d51-4d22-8cbb-c0701c922c3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7606859c-2d51-4d22-8cbb-c0701c922c3b" (UID: "7606859c-2d51-4d22-8cbb-c0701c922c3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:33:56.256952 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:56.256886    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7606859c-2d51-4d22-8cbb-c0701c922c3b-maas-api-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:33:56.951402 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:56.951360    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h" event={"ID":"7606859c-2d51-4d22-8cbb-c0701c922c3b","Type":"ContainerDied","Data":"b4c16d2de292c3d405e6504d65c7094791b0d737662ae89196186f516700eb96"}
Jul 15 20:33:56.951402 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:56.951380    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h"
Jul 15 20:33:56.951682 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:33:56.951417    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4c16d2de292c3d405e6504d65c7094791b0d737662ae89196186f516700eb96"
Jul 15 20:34:27.290750 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:34:27.290616    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:34:27.295917 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:34:27.295746    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:34:27.299807 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:34:27.299787    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:34:27.303065 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:34:27.303044    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:35:01.381667 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.381583    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"]
Jul 15 20:35:01.382704 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.382006    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7606859c-2d51-4d22-8cbb-c0701c922c3b" containerName="revoke-keys"
Jul 15 20:35:01.382704 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.382026    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7606859c-2d51-4d22-8cbb-c0701c922c3b" containerName="revoke-keys"
Jul 15 20:35:01.382704 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.382135    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7606859c-2d51-4d22-8cbb-c0701c922c3b" containerName="revoke-keys"
Jul 15 20:35:01.385320 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.385297    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.387922 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.387887    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"kube-root-ca.crt\""
Jul 15 20:35:01.388833 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.388814    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"auth-test-model-c4ecb2-kserve-self-signed-certs\""
Jul 15 20:35:01.388958 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.388844    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"openshift-service-ca.crt\""
Jul 15 20:35:01.388958 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.388904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"default-dockercfg-2qh5q\""
Jul 15 20:35:01.398482 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.398460    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"]
Jul 15 20:35:01.491778 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.491739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tmp-dir\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.491778 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.491771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tls-certs\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.491995 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.491794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr58c\" (UniqueName: \"kubernetes.io/projected/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-kube-api-access-wr58c\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.491995 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.491881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-model-cache\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.491995 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.491948    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-home\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.491995 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.491968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-dshm\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592347 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tmp-dir\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592347 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tls-certs\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592569 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr58c\" (UniqueName: \"kubernetes.io/projected/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-kube-api-access-wr58c\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592569 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-model-cache\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592569 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-home\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592569 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-dshm\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592858 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tmp-dir\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592944 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-model-cache\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.592944 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.592916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-home\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.594757 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.594737    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-dshm\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.595055 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.595041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tls-certs\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.603691 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.603669    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr58c\" (UniqueName: \"kubernetes.io/projected/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-kube-api-access-wr58c\") pod \"auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") " pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.693946 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.693885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:01.824973 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:01.824948    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"]
Jul 15 20:35:01.827548 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:35:01.827387    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f7b8ae6_8ea2_4cfd_a676_cda576bd74a8.slice/crio-001dc7afe664a893379d174da9a4071627bf0ed67f5a9542a3872b36d52f1c37 WatchSource:0}: Error finding container 001dc7afe664a893379d174da9a4071627bf0ed67f5a9542a3872b36d52f1c37: Status 404 returned error can't find the container with id 001dc7afe664a893379d174da9a4071627bf0ed67f5a9542a3872b36d52f1c37
Jul 15 20:35:02.227312 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:02.227283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7" event={"ID":"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8","Type":"ContainerStarted","Data":"cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c"}
Jul 15 20:35:02.227494 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:02.227321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7" event={"ID":"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8","Type":"ContainerStarted","Data":"001dc7afe664a893379d174da9a4071627bf0ed67f5a9542a3872b36d52f1c37"}
Jul 15 20:35:02.246620 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:02.246576    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7" podStartSLOduration=1.246561374 podStartE2EDuration="1.246561374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:35:02.245569909 +0000 UTC m=+2435.502972085" watchObservedRunningTime="2026-07-15 20:35:02.246561374 +0000 UTC m=+2435.503963594"
Jul 15 20:35:11.694478 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:11.694425    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:11.694478 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:11.694485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:11.706640 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:11.706617    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:12.279673 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:12.279647    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:35:35.696193 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.696153    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"]
Jul 15 20:35:35.699592 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.699569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.702417 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.702395    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"openshift-service-ca.crt\""
Jul 15 20:35:35.703249 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.703227    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"default-dockercfg-xdwr7\""
Jul 15 20:35:35.703340 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.703298    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"auth-test-model-8e78e5-kserve-self-signed-certs\""
Jul 15 20:35:35.703340 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.703296    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"kube-root-ca.crt\""
Jul 15 20:35:35.712542 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.712521    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"]
Jul 15 20:35:35.770672 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.770634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b0678f8-c98c-429f-bc0a-895769c60dd8-tls-certs\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.770862 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.770691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-model-cache\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.770862 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.770746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-dshm\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.770862 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.770791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-644bq\" (UniqueName: \"kubernetes.io/projected/4b0678f8-c98c-429f-bc0a-895769c60dd8-kube-api-access-644bq\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.770862 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.770833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-home\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.770862 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.770852    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-tmp-dir\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.871748 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.871682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-home\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.871748 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.871746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-tmp-dir\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.871961 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.871785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b0678f8-c98c-429f-bc0a-895769c60dd8-tls-certs\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.871961 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.871835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-model-cache\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.871961 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.871856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-dshm\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.871961 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.871879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-644bq\" (UniqueName: \"kubernetes.io/projected/4b0678f8-c98c-429f-bc0a-895769c60dd8-kube-api-access-644bq\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.872175 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.872150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-home\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.872237 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.872164    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-tmp-dir\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.872359 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.872339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-model-cache\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.874288 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.874267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-dshm\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.874455 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.874438    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b0678f8-c98c-429f-bc0a-895769c60dd8-tls-certs\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:35.882969 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:35.882941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-644bq\" (UniqueName: \"kubernetes.io/projected/4b0678f8-c98c-429f-bc0a-895769c60dd8-kube-api-access-644bq\") pod \"auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") " pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:36.008743 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:36.008625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:36.150444 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:36.150415    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"]
Jul 15 20:35:36.153091 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:35:36.153065    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b0678f8_c98c_429f_bc0a_895769c60dd8.slice/crio-2e9c26553f67e72c1003f0c292ad20f5d08b503b22a4c1670a7d6a21c4f9f7b8 WatchSource:0}: Error finding container 2e9c26553f67e72c1003f0c292ad20f5d08b503b22a4c1670a7d6a21c4f9f7b8: Status 404 returned error can't find the container with id 2e9c26553f67e72c1003f0c292ad20f5d08b503b22a4c1670a7d6a21c4f9f7b8
Jul 15 20:35:36.374801 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:36.374761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx" event={"ID":"4b0678f8-c98c-429f-bc0a-895769c60dd8","Type":"ContainerStarted","Data":"222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438"}
Jul 15 20:35:36.374966 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:36.374812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx" event={"ID":"4b0678f8-c98c-429f-bc0a-895769c60dd8","Type":"ContainerStarted","Data":"2e9c26553f67e72c1003f0c292ad20f5d08b503b22a4c1670a7d6a21c4f9f7b8"}
Jul 15 20:35:36.399532 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:36.399477    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx" podStartSLOduration=1.399458654 podStartE2EDuration="1.399458654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:35:36.398264408 +0000 UTC m=+2469.655666581" watchObservedRunningTime="2026-07-15 20:35:36.399458654 +0000 UTC m=+2469.656860833"
Jul 15 20:35:46.009503 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:46.009462    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:46.009503 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:46.009510    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:46.022515 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:46.022488    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:35:46.429903 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:35:46.429876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:37:41.261921 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:41.261888    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"]
Jul 15 20:37:42.183578 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:42.183541    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"]
Jul 15 20:37:56.282407 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.282348    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7" podUID="6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" containerName="main" containerID="cri-o://cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c" gracePeriod=45
Jul 15 20:37:56.526060 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.526035    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:37:56.654421 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654384    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-model-cache\") pod \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") "
Jul 15 20:37:56.654600 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654491    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tmp-dir\") pod \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") "
Jul 15 20:37:56.654600 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654520    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr58c\" (UniqueName: \"kubernetes.io/projected/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-kube-api-access-wr58c\") pod \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") "
Jul 15 20:37:56.654600 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654542    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-home\") pod \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") "
Jul 15 20:37:56.654600 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654567    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tls-certs\") pod \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") "
Jul 15 20:37:56.654600 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654593    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-dshm\") pod \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\" (UID: \"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8\") "
Jul 15 20:37:56.654899 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654698    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" (UID: "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:56.654967 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-home" (OuterVolumeSpecName: "home") pod "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" (UID: "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:56.654967 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654915    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" (UID: "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:56.654967 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.654950    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:56.657118 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.657089    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-dshm" (OuterVolumeSpecName: "dshm") pod "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" (UID: "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:56.657118 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.657103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" (UID: "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:37:56.657251 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.657179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-kube-api-access-wr58c" (OuterVolumeSpecName: "kube-api-access-wr58c") pod "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" (UID: "6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8"). InnerVolumeSpecName "kube-api-access-wr58c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:37:56.755658 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.755624    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:56.755658 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.755652    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr58c\" (UniqueName: \"kubernetes.io/projected/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-kube-api-access-wr58c\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:56.755658 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.755663    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:56.755904 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.755671    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:56.755904 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.755679    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:57.000064 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:56.999977    2563 generic.go:356] "Generic (PLEG): container finished" podID="6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" containerID="cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c" exitCode=0
Jul 15 20:37:57.000064 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.000019    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"
Jul 15 20:37:57.000064 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.000044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7" event={"ID":"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8","Type":"ContainerDied","Data":"cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c"}
Jul 15 20:37:57.000306 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.000077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7" event={"ID":"6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8","Type":"ContainerDied","Data":"001dc7afe664a893379d174da9a4071627bf0ed67f5a9542a3872b36d52f1c37"}
Jul 15 20:37:57.000306 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.000096    2563 scope.go:117] "RemoveContainer" containerID="cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c"
Jul 15 20:37:57.008946 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.008929    2563 scope.go:117] "RemoveContainer" containerID="cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c"
Jul 15 20:37:57.009196 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:37:57.009177    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c\": container with ID starting with cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c not found: ID does not exist" containerID="cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c"
Jul 15 20:37:57.009260 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.009204    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c"} err="failed to get container status \"cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c\": rpc error: code = NotFound desc = could not find container \"cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c\": container with ID starting with cdd7993c19b75e8e90ecf57cbece66932e917d1db20f9541e3de6ace7818456c not found: ID does not exist"
Jul 15 20:37:57.023161 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.023132    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"]
Jul 15 20:37:57.027509 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.027490    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/auth-test-model-c4ecb2-kserve-566c77b8dc-b4xq7"]
Jul 15 20:37:57.203823 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.203755    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx" podUID="4b0678f8-c98c-429f-bc0a-895769c60dd8" containerName="main" containerID="cri-o://222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438" gracePeriod=45
Jul 15 20:37:57.298586 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.298559    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" path="/var/lib/kubelet/pods/6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8/volumes"
Jul 15 20:37:57.443966 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.443944    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:37:57.562298 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562224    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-home\") pod \"4b0678f8-c98c-429f-bc0a-895769c60dd8\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") "
Jul 15 20:37:57.562298 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562283    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-644bq\" (UniqueName: \"kubernetes.io/projected/4b0678f8-c98c-429f-bc0a-895769c60dd8-kube-api-access-644bq\") pod \"4b0678f8-c98c-429f-bc0a-895769c60dd8\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") "
Jul 15 20:37:57.562474 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-tmp-dir\") pod \"4b0678f8-c98c-429f-bc0a-895769c60dd8\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") "
Jul 15 20:37:57.562474 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-dshm\") pod \"4b0678f8-c98c-429f-bc0a-895769c60dd8\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") "
Jul 15 20:37:57.562474 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562400    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b0678f8-c98c-429f-bc0a-895769c60dd8-tls-certs\") pod \"4b0678f8-c98c-429f-bc0a-895769c60dd8\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") "
Jul 15 20:37:57.562474 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562428    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-model-cache\") pod \"4b0678f8-c98c-429f-bc0a-895769c60dd8\" (UID: \"4b0678f8-c98c-429f-bc0a-895769c60dd8\") "
Jul 15 20:37:57.562474 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562363    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-home" (OuterVolumeSpecName: "home") pod "4b0678f8-c98c-429f-bc0a-895769c60dd8" (UID: "4b0678f8-c98c-429f-bc0a-895769c60dd8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:57.562687 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562551    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b0678f8-c98c-429f-bc0a-895769c60dd8" (UID: "4b0678f8-c98c-429f-bc0a-895769c60dd8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:57.562770 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562683    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:57.562770 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562756    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:57.562855 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.562775    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b0678f8-c98c-429f-bc0a-895769c60dd8" (UID: "4b0678f8-c98c-429f-bc0a-895769c60dd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:57.564599 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.564577    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-dshm" (OuterVolumeSpecName: "dshm") pod "4b0678f8-c98c-429f-bc0a-895769c60dd8" (UID: "4b0678f8-c98c-429f-bc0a-895769c60dd8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:37:57.564675 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.564654    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0678f8-c98c-429f-bc0a-895769c60dd8-kube-api-access-644bq" (OuterVolumeSpecName: "kube-api-access-644bq") pod "4b0678f8-c98c-429f-bc0a-895769c60dd8" (UID: "4b0678f8-c98c-429f-bc0a-895769c60dd8"). InnerVolumeSpecName "kube-api-access-644bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:37:57.564714 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.564659    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b0678f8-c98c-429f-bc0a-895769c60dd8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b0678f8-c98c-429f-bc0a-895769c60dd8" (UID: "4b0678f8-c98c-429f-bc0a-895769c60dd8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:37:57.663123 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.663095    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:57.663123 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.663118    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b0678f8-c98c-429f-bc0a-895769c60dd8-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:57.663123 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.663127    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b0678f8-c98c-429f-bc0a-895769c60dd8-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:57.663314 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:57.663136    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-644bq\" (UniqueName: \"kubernetes.io/projected/4b0678f8-c98c-429f-bc0a-895769c60dd8-kube-api-access-644bq\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:37:58.005538 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.005504    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b0678f8-c98c-429f-bc0a-895769c60dd8" containerID="222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438" exitCode=0
Jul 15 20:37:58.005764 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.005547    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"
Jul 15 20:37:58.005764 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.005563    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx" event={"ID":"4b0678f8-c98c-429f-bc0a-895769c60dd8","Type":"ContainerDied","Data":"222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438"}
Jul 15 20:37:58.005764 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.005612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx" event={"ID":"4b0678f8-c98c-429f-bc0a-895769c60dd8","Type":"ContainerDied","Data":"2e9c26553f67e72c1003f0c292ad20f5d08b503b22a4c1670a7d6a21c4f9f7b8"}
Jul 15 20:37:58.005764 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.005635    2563 scope.go:117] "RemoveContainer" containerID="222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438"
Jul 15 20:37:58.014515 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.014498    2563 scope.go:117] "RemoveContainer" containerID="222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438"
Jul 15 20:37:58.014802 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:37:58.014784    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438\": container with ID starting with 222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438 not found: ID does not exist" containerID="222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438"
Jul 15 20:37:58.014878 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.014811    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438"} err="failed to get container status \"222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438\": rpc error: code = NotFound desc = could not find container \"222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438\": container with ID starting with 222b4dcc3868ce0f1b060d2ea469847cd8d3ab495ee4b9a7dbc46d4a4ca35438 not found: ID does not exist"
Jul 15 20:37:58.028038 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.028015    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"]
Jul 15 20:37:58.032536 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:58.032515    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/auth-test-model-8e78e5-kserve-c7d86b9bc-g8hkx"]
Jul 15 20:37:59.297347 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:37:59.297310    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b0678f8-c98c-429f-bc0a-895769c60dd8" path="/var/lib/kubelet/pods/4b0678f8-c98c-429f-bc0a-895769c60dd8/volumes"
Jul 15 20:38:14.272659 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.272616    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"]
Jul 15 20:38:14.273257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.273059    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0678f8-c98c-429f-bc0a-895769c60dd8" containerName="main"
Jul 15 20:38:14.273257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.273078    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0678f8-c98c-429f-bc0a-895769c60dd8" containerName="main"
Jul 15 20:38:14.273257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.273103    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" containerName="main"
Jul 15 20:38:14.273257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.273111    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" containerName="main"
Jul 15 20:38:14.273257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.273186    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7b8ae6-8ea2-4cfd-a676-cda576bd74a8" containerName="main"
Jul 15 20:38:14.273257 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.273196    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0678f8-c98c-429f-bc0a-895769c60dd8" containerName="main"
Jul 15 20:38:14.275947 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.275927    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.279396 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.279367    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"kube-root-ca.crt\""
Jul 15 20:38:14.279509 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.279404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"sub-test-model-8e78e5-kserve-self-signed-certs\""
Jul 15 20:38:14.279509 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.279424    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"openshift-service-ca.crt\""
Jul 15 20:38:14.279509 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.279367    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8e78e5\"/\"default-dockercfg-xdwr7\""
Jul 15 20:38:14.287055 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.287024    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"]
Jul 15 20:38:14.400119 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.400084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-dshm\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.400119 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.400119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-tmp-dir\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.400318 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.400174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-model-cache\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.400383 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.400365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7077f872-5205-4ece-955f-92fba7407bcc-tls-certs\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.400423 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.400400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8dnb\" (UniqueName: \"kubernetes.io/projected/7077f872-5205-4ece-955f-92fba7407bcc-kube-api-access-d8dnb\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.400456 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.400440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-home\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501185 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-home\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501322 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-dshm\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501322 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-tmp-dir\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501322 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-model-cache\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501322 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7077f872-5205-4ece-955f-92fba7407bcc-tls-certs\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501322 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8dnb\" (UniqueName: \"kubernetes.io/projected/7077f872-5205-4ece-955f-92fba7407bcc-kube-api-access-d8dnb\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501562 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-home\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501665 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-model-cache\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.501755 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.501691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-tmp-dir\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.503624 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.503603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-dshm\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.503891 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.503875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7077f872-5205-4ece-955f-92fba7407bcc-tls-certs\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.510266 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.510240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8dnb\" (UniqueName: \"kubernetes.io/projected/7077f872-5205-4ece-955f-92fba7407bcc-kube-api-access-d8dnb\") pod \"sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") " pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.584915 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.584817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:14.716744 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:14.716698    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"]
Jul 15 20:38:14.719160 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:38:14.719131    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7077f872_5205_4ece_955f_92fba7407bcc.slice/crio-93bf3e99855978ed4d57a352974a8f57925e20e55a75b8dcce3c222b51fb6b48 WatchSource:0}: Error finding container 93bf3e99855978ed4d57a352974a8f57925e20e55a75b8dcce3c222b51fb6b48: Status 404 returned error can't find the container with id 93bf3e99855978ed4d57a352974a8f57925e20e55a75b8dcce3c222b51fb6b48
Jul 15 20:38:15.085551 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:15.085515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5" event={"ID":"7077f872-5205-4ece-955f-92fba7407bcc","Type":"ContainerStarted","Data":"cddfcbc0c68550b19813342ffa3bc3b66297fbc56f758c5d19c316cb5723b135"}
Jul 15 20:38:15.085754 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:15.085560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5" event={"ID":"7077f872-5205-4ece-955f-92fba7407bcc","Type":"ContainerStarted","Data":"93bf3e99855978ed4d57a352974a8f57925e20e55a75b8dcce3c222b51fb6b48"}
Jul 15 20:38:15.103103 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:15.103063    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5" podStartSLOduration=1.103047289 podStartE2EDuration="1.103047289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:38:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:38:15.102433007 +0000 UTC m=+2628.359835186" watchObservedRunningTime="2026-07-15 20:38:15.103047289 +0000 UTC m=+2628.360449468"
Jul 15 20:38:24.585056 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:24.585012    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:24.585441 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:24.585069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:24.597739 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:24.597698    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:38:25.140261 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:38:25.140235    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:39:03.172351 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:03.172320    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"]
Jul 15 20:39:18.191855 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.191650    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5" podUID="7077f872-5205-4ece-955f-92fba7407bcc" containerName="main" containerID="cri-o://cddfcbc0c68550b19813342ffa3bc3b66297fbc56f758c5d19c316cb5723b135" gracePeriod=45
Jul 15 20:39:18.365890 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.365858    2563 generic.go:356] "Generic (PLEG): container finished" podID="7077f872-5205-4ece-955f-92fba7407bcc" containerID="cddfcbc0c68550b19813342ffa3bc3b66297fbc56f758c5d19c316cb5723b135" exitCode=0
Jul 15 20:39:18.366008 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.365921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5" event={"ID":"7077f872-5205-4ece-955f-92fba7407bcc","Type":"ContainerDied","Data":"cddfcbc0c68550b19813342ffa3bc3b66297fbc56f758c5d19c316cb5723b135"}
Jul 15 20:39:18.442027 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.441977    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:39:18.532281 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532246    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-model-cache\") pod \"7077f872-5205-4ece-955f-92fba7407bcc\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") "
Jul 15 20:39:18.532281 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532286    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7077f872-5205-4ece-955f-92fba7407bcc-tls-certs\") pod \"7077f872-5205-4ece-955f-92fba7407bcc\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") "
Jul 15 20:39:18.532528 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-tmp-dir\") pod \"7077f872-5205-4ece-955f-92fba7407bcc\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") "
Jul 15 20:39:18.532528 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532357    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-dshm\") pod \"7077f872-5205-4ece-955f-92fba7407bcc\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") "
Jul 15 20:39:18.532528 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532373    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8dnb\" (UniqueName: \"kubernetes.io/projected/7077f872-5205-4ece-955f-92fba7407bcc-kube-api-access-d8dnb\") pod \"7077f872-5205-4ece-955f-92fba7407bcc\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") "
Jul 15 20:39:18.532528 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532399    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-home\") pod \"7077f872-5205-4ece-955f-92fba7407bcc\" (UID: \"7077f872-5205-4ece-955f-92fba7407bcc\") "
Jul 15 20:39:18.532757 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532579    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-model-cache" (OuterVolumeSpecName: "model-cache") pod "7077f872-5205-4ece-955f-92fba7407bcc" (UID: "7077f872-5205-4ece-955f-92fba7407bcc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:39:18.532757 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532590    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7077f872-5205-4ece-955f-92fba7407bcc" (UID: "7077f872-5205-4ece-955f-92fba7407bcc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:39:18.532757 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532748    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:39:18.532916 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532768    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:39:18.532916 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.532798    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-home" (OuterVolumeSpecName: "home") pod "7077f872-5205-4ece-955f-92fba7407bcc" (UID: "7077f872-5205-4ece-955f-92fba7407bcc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:39:18.534608 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.534581    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-dshm" (OuterVolumeSpecName: "dshm") pod "7077f872-5205-4ece-955f-92fba7407bcc" (UID: "7077f872-5205-4ece-955f-92fba7407bcc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:39:18.534713 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.534609    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7077f872-5205-4ece-955f-92fba7407bcc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7077f872-5205-4ece-955f-92fba7407bcc" (UID: "7077f872-5205-4ece-955f-92fba7407bcc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:39:18.534713 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.534682    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7077f872-5205-4ece-955f-92fba7407bcc-kube-api-access-d8dnb" (OuterVolumeSpecName: "kube-api-access-d8dnb") pod "7077f872-5205-4ece-955f-92fba7407bcc" (UID: "7077f872-5205-4ece-955f-92fba7407bcc"). InnerVolumeSpecName "kube-api-access-d8dnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:39:18.633162 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.633128    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7077f872-5205-4ece-955f-92fba7407bcc-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:39:18.633162 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.633156    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:39:18.633162 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.633165    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8dnb\" (UniqueName: \"kubernetes.io/projected/7077f872-5205-4ece-955f-92fba7407bcc-kube-api-access-d8dnb\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:39:18.633371 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:18.633174    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7077f872-5205-4ece-955f-92fba7407bcc-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:39:19.371443 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:19.371418    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"
Jul 15 20:39:19.371875 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:19.371452    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5" event={"ID":"7077f872-5205-4ece-955f-92fba7407bcc","Type":"ContainerDied","Data":"93bf3e99855978ed4d57a352974a8f57925e20e55a75b8dcce3c222b51fb6b48"}
Jul 15 20:39:19.371875 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:19.371493    2563 scope.go:117] "RemoveContainer" containerID="cddfcbc0c68550b19813342ffa3bc3b66297fbc56f758c5d19c316cb5723b135"
Jul 15 20:39:19.392461 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:19.392436    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"]
Jul 15 20:39:19.394071 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:19.394048    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8e78e5/sub-test-model-8e78e5-kserve-5cff4476cf-9nfc5"]
Jul 15 20:39:21.296501 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.296465    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7077f872-5205-4ece-955f-92fba7407bcc" path="/var/lib/kubelet/pods/7077f872-5205-4ece-955f-92fba7407bcc/volumes"
Jul 15 20:39:21.872146 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.872113    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"]
Jul 15 20:39:21.872431 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.872418    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7077f872-5205-4ece-955f-92fba7407bcc" containerName="main"
Jul 15 20:39:21.872473 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.872436    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7077f872-5205-4ece-955f-92fba7407bcc" containerName="main"
Jul 15 20:39:21.872513 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.872507    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7077f872-5205-4ece-955f-92fba7407bcc" containerName="main"
Jul 15 20:39:21.876938 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.876922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:21.879406 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.879383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"default-dockercfg-2qh5q\""
Jul 15 20:39:21.879536 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.879419    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"openshift-service-ca.crt\""
Jul 15 20:39:21.880518 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.880492    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"kube-root-ca.crt\""
Jul 15 20:39:21.880518 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.880503    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4ecb2\"/\"rate-test-model-c4ecb2-kserve-self-signed-certs\""
Jul 15 20:39:21.885422 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.885400    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"]
Jul 15 20:39:21.955906 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.955876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-tmp-dir\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:21.956048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.955914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-dshm\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:21.956048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.955983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98cad66-4204-475b-8094-93793dcc43ad-tls-certs\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:21.956048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.956023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-home\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:21.956177 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.956053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2pxc\" (UniqueName: \"kubernetes.io/projected/b98cad66-4204-475b-8094-93793dcc43ad-kube-api-access-z2pxc\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:21.956177 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:21.956135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-model-cache\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.056525 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.056492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2pxc\" (UniqueName: \"kubernetes.io/projected/b98cad66-4204-475b-8094-93793dcc43ad-kube-api-access-z2pxc\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.056684 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.056546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-model-cache\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.056684 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.056579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-tmp-dir\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.056684 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.056604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-dshm\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.056870 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.056769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98cad66-4204-475b-8094-93793dcc43ad-tls-certs\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.056870 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.056812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-home\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.057096 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.057074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-tmp-dir\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.057096 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.057089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-model-cache\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.057172 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.057136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-home\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.058945 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.058925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-dshm\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.059220 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.059202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98cad66-4204-475b-8094-93793dcc43ad-tls-certs\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.064424 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.064403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2pxc\" (UniqueName: \"kubernetes.io/projected/b98cad66-4204-475b-8094-93793dcc43ad-kube-api-access-z2pxc\") pod \"rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") " pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.186503 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.186427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:22.314889 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.314859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"]
Jul 15 20:39:22.318020 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:39:22.317997    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb98cad66_4204_475b_8094_93793dcc43ad.slice/crio-a89c29fcbe3e53dd909cba2814e4c0daa7d459979692458537395aa0d10a1326 WatchSource:0}: Error finding container a89c29fcbe3e53dd909cba2814e4c0daa7d459979692458537395aa0d10a1326: Status 404 returned error can't find the container with id a89c29fcbe3e53dd909cba2814e4c0daa7d459979692458537395aa0d10a1326
Jul 15 20:39:22.385539 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.385507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj" event={"ID":"b98cad66-4204-475b-8094-93793dcc43ad","Type":"ContainerStarted","Data":"06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666"}
Jul 15 20:39:22.385651 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.385544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj" event={"ID":"b98cad66-4204-475b-8094-93793dcc43ad","Type":"ContainerStarted","Data":"a89c29fcbe3e53dd909cba2814e4c0daa7d459979692458537395aa0d10a1326"}
Jul 15 20:39:22.404113 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:22.403524    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj" podStartSLOduration=1.403507184 podStartE2EDuration="1.403507184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:39:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:39:22.403229142 +0000 UTC m=+2695.660631320" watchObservedRunningTime="2026-07-15 20:39:22.403507184 +0000 UTC m=+2695.660909364"
Jul 15 20:39:27.315995 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:27.315887    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:39:27.321282 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:27.321263    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:39:27.323211 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:27.323194    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:39:27.327934 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:27.327919    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sfcd_42abef4d-d44e-48a7-8847-904feec67e60/ovn-acl-logging/0.log"
Jul 15 20:39:32.187167 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:32.187132    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:32.187542 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:32.187191    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:32.199387 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:32.199364    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:39:32.438264 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:39:32.438186    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:40:33.889864 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:33.889832    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"]
Jul 15 20:40:35.635661 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.635626    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7367-139800200780208"]
Jul 15 20:40:35.639032 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.639015    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780208"
Jul 15 20:40:35.641836 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.641815    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8flk9\""
Jul 15 20:40:35.647364 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.647341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200780208"]
Jul 15 20:40:35.667082 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.667051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6mmf\" (UniqueName: \"kubernetes.io/projected/af83c4f4-78f3-432b-9a00-23ad5a731d67-kube-api-access-l6mmf\") pod \"test-curl-7367-139800200780208\" (UID: \"af83c4f4-78f3-432b-9a00-23ad5a731d67\") " pod="opendatahub/test-curl-7367-139800200780208"
Jul 15 20:40:35.768063 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.768035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6mmf\" (UniqueName: \"kubernetes.io/projected/af83c4f4-78f3-432b-9a00-23ad5a731d67-kube-api-access-l6mmf\") pod \"test-curl-7367-139800200780208\" (UID: \"af83c4f4-78f3-432b-9a00-23ad5a731d67\") " pod="opendatahub/test-curl-7367-139800200780208"
Jul 15 20:40:35.776308 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.776283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6mmf\" (UniqueName: \"kubernetes.io/projected/af83c4f4-78f3-432b-9a00-23ad5a731d67-kube-api-access-l6mmf\") pod \"test-curl-7367-139800200780208\" (UID: \"af83c4f4-78f3-432b-9a00-23ad5a731d67\") " pod="opendatahub/test-curl-7367-139800200780208"
Jul 15 20:40:35.948646 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:35.948550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780208"
Jul 15 20:40:36.089796 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:36.089772    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200780208"]
Jul 15 20:40:36.091767 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:40:36.091736    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf83c4f4_78f3_432b_9a00_23ad5a731d67.slice/crio-290373bc6c8647f762fb83c05fe57cab2994cfaace46fe4eea781fd565f19659 WatchSource:0}: Error finding container 290373bc6c8647f762fb83c05fe57cab2994cfaace46fe4eea781fd565f19659: Status 404 returned error can't find the container with id 290373bc6c8647f762fb83c05fe57cab2994cfaace46fe4eea781fd565f19659
Jul 15 20:40:36.093043 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:36.093024    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:40:36.706044 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:36.706013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200780208" event={"ID":"af83c4f4-78f3-432b-9a00-23ad5a731d67","Type":"ContainerStarted","Data":"290373bc6c8647f762fb83c05fe57cab2994cfaace46fe4eea781fd565f19659"}
Jul 15 20:40:37.711627 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:37.711585    2563 generic.go:356] "Generic (PLEG): container finished" podID="af83c4f4-78f3-432b-9a00-23ad5a731d67" containerID="b63938bb3249ed84291e54637837ff805b479b98a08a2f0485b0cf80fbc4443c" exitCode=0
Jul 15 20:40:37.712027 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:37.711646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200780208" event={"ID":"af83c4f4-78f3-432b-9a00-23ad5a731d67","Type":"ContainerDied","Data":"b63938bb3249ed84291e54637837ff805b479b98a08a2f0485b0cf80fbc4443c"}
Jul 15 20:40:38.845868 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:38.845843    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780208"
Jul 15 20:40:38.885197 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:38.885168    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7367-139800200780208_af83c4f4-78f3-432b-9a00-23ad5a731d67/test-curl-7367-139800200780208/0.log"
Jul 15 20:40:38.893400 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:38.893373    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6mmf\" (UniqueName: \"kubernetes.io/projected/af83c4f4-78f3-432b-9a00-23ad5a731d67-kube-api-access-l6mmf\") pod \"af83c4f4-78f3-432b-9a00-23ad5a731d67\" (UID: \"af83c4f4-78f3-432b-9a00-23ad5a731d67\") "
Jul 15 20:40:38.895538 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:38.895512    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af83c4f4-78f3-432b-9a00-23ad5a731d67-kube-api-access-l6mmf" (OuterVolumeSpecName: "kube-api-access-l6mmf") pod "af83c4f4-78f3-432b-9a00-23ad5a731d67" (UID: "af83c4f4-78f3-432b-9a00-23ad5a731d67"). InnerVolumeSpecName "kube-api-access-l6mmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:40:38.912815 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:38.912781    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7367-139800200780208"]
Jul 15 20:40:38.916732 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:38.916694    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7367-139800200780208"]
Jul 15 20:40:38.994673 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:38.994600    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6mmf\" (UniqueName: \"kubernetes.io/projected/af83c4f4-78f3-432b-9a00-23ad5a731d67-kube-api-access-l6mmf\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:39.042221 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.042181    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7367-139800200780464"]
Jul 15 20:40:39.042516 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.042502    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af83c4f4-78f3-432b-9a00-23ad5a731d67" containerName="test-curl-7367-139800200780208"
Jul 15 20:40:39.042565 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.042518    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af83c4f4-78f3-432b-9a00-23ad5a731d67" containerName="test-curl-7367-139800200780208"
Jul 15 20:40:39.042598 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.042583    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af83c4f4-78f3-432b-9a00-23ad5a731d67" containerName="test-curl-7367-139800200780208"
Jul 15 20:40:39.045464 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.045446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:39.051310 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.051280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200780464"]
Jul 15 20:40:39.095507 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.095463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl4h5\" (UniqueName: \"kubernetes.io/projected/257d026c-47fa-4f7e-b0e4-988bd2ea6ef0-kube-api-access-xl4h5\") pod \"test-curl-7367-139800200780464\" (UID: \"257d026c-47fa-4f7e-b0e4-988bd2ea6ef0\") " pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:39.196511 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.196464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl4h5\" (UniqueName: \"kubernetes.io/projected/257d026c-47fa-4f7e-b0e4-988bd2ea6ef0-kube-api-access-xl4h5\") pod \"test-curl-7367-139800200780464\" (UID: \"257d026c-47fa-4f7e-b0e4-988bd2ea6ef0\") " pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:39.205956 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.205925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl4h5\" (UniqueName: \"kubernetes.io/projected/257d026c-47fa-4f7e-b0e4-988bd2ea6ef0-kube-api-access-xl4h5\") pod \"test-curl-7367-139800200780464\" (UID: \"257d026c-47fa-4f7e-b0e4-988bd2ea6ef0\") " pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:39.297060 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.296980    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af83c4f4-78f3-432b-9a00-23ad5a731d67" path="/var/lib/kubelet/pods/af83c4f4-78f3-432b-9a00-23ad5a731d67/volumes"
Jul 15 20:40:39.354884 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.354851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:39.497659 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.497635    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200780464"]
Jul 15 20:40:39.499536 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:40:39.499513    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod257d026c_47fa_4f7e_b0e4_988bd2ea6ef0.slice/crio-7fb3a9df1bf5ff52a66b6340ba4f1ccf93fb1f282bdf99ee0568f4a525d9d238 WatchSource:0}: Error finding container 7fb3a9df1bf5ff52a66b6340ba4f1ccf93fb1f282bdf99ee0568f4a525d9d238: Status 404 returned error can't find the container with id 7fb3a9df1bf5ff52a66b6340ba4f1ccf93fb1f282bdf99ee0568f4a525d9d238
Jul 15 20:40:39.721785 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.721753    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780208"
Jul 15 20:40:39.721785 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.721781    2563 scope.go:117] "RemoveContainer" containerID="b63938bb3249ed84291e54637837ff805b479b98a08a2f0485b0cf80fbc4443c"
Jul 15 20:40:39.723146 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:39.723011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200780464" event={"ID":"257d026c-47fa-4f7e-b0e4-988bd2ea6ef0","Type":"ContainerStarted","Data":"7fb3a9df1bf5ff52a66b6340ba4f1ccf93fb1f282bdf99ee0568f4a525d9d238"}
Jul 15 20:40:40.730251 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:40.730214    2563 generic.go:356] "Generic (PLEG): container finished" podID="257d026c-47fa-4f7e-b0e4-988bd2ea6ef0" containerID="7e12e74f1395a2296b505ec8ab9a850c462a44b43c55e951cc9b46add649d0f4" exitCode=0
Jul 15 20:40:40.730691 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:40.730299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200780464" event={"ID":"257d026c-47fa-4f7e-b0e4-988bd2ea6ef0","Type":"ContainerDied","Data":"7e12e74f1395a2296b505ec8ab9a850c462a44b43c55e951cc9b46add649d0f4"}
Jul 15 20:40:41.862562 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:41.862541    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:41.885714 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:41.885690    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7367-139800200780464_257d026c-47fa-4f7e-b0e4-988bd2ea6ef0/test-curl-7367-139800200780464/0.log"
Jul 15 20:40:41.911886 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:41.911856    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7367-139800200780464"]
Jul 15 20:40:41.915951 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:41.915930    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7367-139800200780464"]
Jul 15 20:40:41.920473 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:41.920454    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl4h5\" (UniqueName: \"kubernetes.io/projected/257d026c-47fa-4f7e-b0e4-988bd2ea6ef0-kube-api-access-xl4h5\") pod \"257d026c-47fa-4f7e-b0e4-988bd2ea6ef0\" (UID: \"257d026c-47fa-4f7e-b0e4-988bd2ea6ef0\") "
Jul 15 20:40:41.922649 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:41.922623    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/257d026c-47fa-4f7e-b0e4-988bd2ea6ef0-kube-api-access-xl4h5" (OuterVolumeSpecName: "kube-api-access-xl4h5") pod "257d026c-47fa-4f7e-b0e4-988bd2ea6ef0" (UID: "257d026c-47fa-4f7e-b0e4-988bd2ea6ef0"). InnerVolumeSpecName "kube-api-access-xl4h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:40:42.021946 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.021868    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl4h5\" (UniqueName: \"kubernetes.io/projected/257d026c-47fa-4f7e-b0e4-988bd2ea6ef0-kube-api-access-xl4h5\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:42.069544 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.069514    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:42.069854 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.069829    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="257d026c-47fa-4f7e-b0e4-988bd2ea6ef0" containerName="test-curl-7367-139800200780464"
Jul 15 20:40:42.069854 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.069856    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="257d026c-47fa-4f7e-b0e4-988bd2ea6ef0" containerName="test-curl-7367-139800200780464"
Jul 15 20:40:42.069940 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.069920    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="257d026c-47fa-4f7e-b0e4-988bd2ea6ef0" containerName="test-curl-7367-139800200780464"
Jul 15 20:40:42.072520 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.072505    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:42.079678 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.079655    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:42.122791 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.122765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rvlf\" (UniqueName: \"kubernetes.io/projected/38a65d11-e64c-4233-9ea2-00c053ef404b-kube-api-access-5rvlf\") pod \"test-curl-7367-139800200790064\" (UID: \"38a65d11-e64c-4233-9ea2-00c053ef404b\") " pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:42.223587 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.223557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvlf\" (UniqueName: \"kubernetes.io/projected/38a65d11-e64c-4233-9ea2-00c053ef404b-kube-api-access-5rvlf\") pod \"test-curl-7367-139800200790064\" (UID: \"38a65d11-e64c-4233-9ea2-00c053ef404b\") " pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:42.234777 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.234748    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvlf\" (UniqueName: \"kubernetes.io/projected/38a65d11-e64c-4233-9ea2-00c053ef404b-kube-api-access-5rvlf\") pod \"test-curl-7367-139800200790064\" (UID: \"38a65d11-e64c-4233-9ea2-00c053ef404b\") " pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:42.382535 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.382502    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:42.509411 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.509368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:42.511403 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:40:42.511375    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38a65d11_e64c_4233_9ea2_00c053ef404b.slice/crio-bd9ff90963120899adfa8e7343d06964a3766a29e63a3324e548f0eb67ef3340 WatchSource:0}: Error finding container bd9ff90963120899adfa8e7343d06964a3766a29e63a3324e548f0eb67ef3340: Status 404 returned error can't find the container with id bd9ff90963120899adfa8e7343d06964a3766a29e63a3324e548f0eb67ef3340
Jul 15 20:40:42.739956 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.739855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200790064" event={"ID":"38a65d11-e64c-4233-9ea2-00c053ef404b","Type":"ContainerStarted","Data":"bd9ff90963120899adfa8e7343d06964a3766a29e63a3324e548f0eb67ef3340"}
Jul 15 20:40:42.741006 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.740987    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fb3a9df1bf5ff52a66b6340ba4f1ccf93fb1f282bdf99ee0568f4a525d9d238"
Jul 15 20:40:42.741122 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:42.741018    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:42.750934 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:40:42.750908    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7367-139800200780464\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="257d026c-47fa-4f7e-b0e4-988bd2ea6ef0" pod="opendatahub/test-curl-7367-139800200780464"
Jul 15 20:40:43.297514 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:43.297480    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="257d026c-47fa-4f7e-b0e4-988bd2ea6ef0" path="/var/lib/kubelet/pods/257d026c-47fa-4f7e-b0e4-988bd2ea6ef0/volumes"
Jul 15 20:40:43.746670 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:43.746640    2563 generic.go:356] "Generic (PLEG): container finished" podID="38a65d11-e64c-4233-9ea2-00c053ef404b" containerID="699dc82a995305c61b3ea241eb481f826d991eeb75838147c688b0d6f6057810" exitCode=0
Jul 15 20:40:43.746870 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:43.746745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200790064" event={"ID":"38a65d11-e64c-4233-9ea2-00c053ef404b","Type":"ContainerDied","Data":"699dc82a995305c61b3ea241eb481f826d991eeb75838147c688b0d6f6057810"}
Jul 15 20:40:44.876352 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:44.876331    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:44.896556 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:44.896529    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7367-139800200790064_38a65d11-e64c-4233-9ea2-00c053ef404b/test-curl-7367-139800200790064/0.log"
Jul 15 20:40:44.917482 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:44.917454    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:44.922641 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:44.922617    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:44.945688 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:44.945659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rvlf\" (UniqueName: \"kubernetes.io/projected/38a65d11-e64c-4233-9ea2-00c053ef404b-kube-api-access-5rvlf\") pod \"38a65d11-e64c-4233-9ea2-00c053ef404b\" (UID: \"38a65d11-e64c-4233-9ea2-00c053ef404b\") "
Jul 15 20:40:44.947780 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:44.947759    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a65d11-e64c-4233-9ea2-00c053ef404b-kube-api-access-5rvlf" (OuterVolumeSpecName: "kube-api-access-5rvlf") pod "38a65d11-e64c-4233-9ea2-00c053ef404b" (UID: "38a65d11-e64c-4233-9ea2-00c053ef404b"). InnerVolumeSpecName "kube-api-access-5rvlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:40:45.046325 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.046237    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rvlf\" (UniqueName: \"kubernetes.io/projected/38a65d11-e64c-4233-9ea2-00c053ef404b-kube-api-access-5rvlf\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:45.058251 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.058218    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:45.058506 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.058493    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a65d11-e64c-4233-9ea2-00c053ef404b" containerName="test-curl-7367-139800200790064"
Jul 15 20:40:45.058550 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.058508    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a65d11-e64c-4233-9ea2-00c053ef404b" containerName="test-curl-7367-139800200790064"
Jul 15 20:40:45.058582 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.058568    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a65d11-e64c-4233-9ea2-00c053ef404b" containerName="test-curl-7367-139800200790064"
Jul 15 20:40:45.061368 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.061353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:45.067709 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.067686    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:45.146961 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.146930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg9x9\" (UniqueName: \"kubernetes.io/projected/6a2740be-f0d1-4af5-bdfd-241be8c35184-kube-api-access-tg9x9\") pod \"test-curl-7367-139800200790064\" (UID: \"6a2740be-f0d1-4af5-bdfd-241be8c35184\") " pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:45.247371 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.247341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg9x9\" (UniqueName: \"kubernetes.io/projected/6a2740be-f0d1-4af5-bdfd-241be8c35184-kube-api-access-tg9x9\") pod \"test-curl-7367-139800200790064\" (UID: \"6a2740be-f0d1-4af5-bdfd-241be8c35184\") " pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:45.256400 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.256374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg9x9\" (UniqueName: \"kubernetes.io/projected/6a2740be-f0d1-4af5-bdfd-241be8c35184-kube-api-access-tg9x9\") pod \"test-curl-7367-139800200790064\" (UID: \"6a2740be-f0d1-4af5-bdfd-241be8c35184\") " pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:45.299063 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.298984    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a65d11-e64c-4233-9ea2-00c053ef404b" path="/var/lib/kubelet/pods/38a65d11-e64c-4233-9ea2-00c053ef404b/volumes"
Jul 15 20:40:45.370152 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.370120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:45.502209 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.502183    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:45.504587 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:40:45.504557    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a2740be_f0d1_4af5_bdfd_241be8c35184.slice/crio-0e51712aa83633fb2699bf31e0ac7995ee4c0e5fc5a37bc980d136d8d9a784d1 WatchSource:0}: Error finding container 0e51712aa83633fb2699bf31e0ac7995ee4c0e5fc5a37bc980d136d8d9a784d1: Status 404 returned error can't find the container with id 0e51712aa83633fb2699bf31e0ac7995ee4c0e5fc5a37bc980d136d8d9a784d1
Jul 15 20:40:45.756977 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.756946    2563 scope.go:117] "RemoveContainer" containerID="699dc82a995305c61b3ea241eb481f826d991eeb75838147c688b0d6f6057810"
Jul 15 20:40:45.757162 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.756947    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:45.758153 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:45.758113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200790064" event={"ID":"6a2740be-f0d1-4af5-bdfd-241be8c35184","Type":"ContainerStarted","Data":"0e51712aa83633fb2699bf31e0ac7995ee4c0e5fc5a37bc980d136d8d9a784d1"}
Jul 15 20:40:46.765664 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:46.765631    2563 generic.go:356] "Generic (PLEG): container finished" podID="6a2740be-f0d1-4af5-bdfd-241be8c35184" containerID="e3df5f3fbd012ce3014248506f8cc92a4020d6fd2f239e97773cbbff0cd2a193" exitCode=0
Jul 15 20:40:46.766064 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:46.765700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200790064" event={"ID":"6a2740be-f0d1-4af5-bdfd-241be8c35184","Type":"ContainerDied","Data":"e3df5f3fbd012ce3014248506f8cc92a4020d6fd2f239e97773cbbff0cd2a193"}
Jul 15 20:40:47.898896 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:47.898872    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:47.920599 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:47.920571    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7367-139800200790064_6a2740be-f0d1-4af5-bdfd-241be8c35184/test-curl-7367-139800200790064/0.log"
Jul 15 20:40:47.948516 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:47.948481    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:47.950920 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:47.950900    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7367-139800200790064"]
Jul 15 20:40:47.971238 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:47.971218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg9x9\" (UniqueName: \"kubernetes.io/projected/6a2740be-f0d1-4af5-bdfd-241be8c35184-kube-api-access-tg9x9\") pod \"6a2740be-f0d1-4af5-bdfd-241be8c35184\" (UID: \"6a2740be-f0d1-4af5-bdfd-241be8c35184\") "
Jul 15 20:40:47.973361 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:47.973337    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2740be-f0d1-4af5-bdfd-241be8c35184-kube-api-access-tg9x9" (OuterVolumeSpecName: "kube-api-access-tg9x9") pod "6a2740be-f0d1-4af5-bdfd-241be8c35184" (UID: "6a2740be-f0d1-4af5-bdfd-241be8c35184"). InnerVolumeSpecName "kube-api-access-tg9x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:40:48.069585 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.069515    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:48.069825 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.069812    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2740be-f0d1-4af5-bdfd-241be8c35184" containerName="test-curl-7367-139800200790064"
Jul 15 20:40:48.069886 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.069826    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2740be-f0d1-4af5-bdfd-241be8c35184" containerName="test-curl-7367-139800200790064"
Jul 15 20:40:48.069886 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.069877    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2740be-f0d1-4af5-bdfd-241be8c35184" containerName="test-curl-7367-139800200790064"
Jul 15 20:40:48.072182 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.072164    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg9x9\" (UniqueName: \"kubernetes.io/projected/6a2740be-f0d1-4af5-bdfd-241be8c35184-kube-api-access-tg9x9\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:48.072646 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.072631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:48.080056 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.080037    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:48.173638 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.173594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkf24\" (UniqueName: \"kubernetes.io/projected/7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351-kube-api-access-mkf24\") pod \"test-curl-7367-139800200791072\" (UID: \"7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351\") " pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:48.274924 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.274885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkf24\" (UniqueName: \"kubernetes.io/projected/7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351-kube-api-access-mkf24\") pod \"test-curl-7367-139800200791072\" (UID: \"7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351\") " pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:48.283365 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.283333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkf24\" (UniqueName: \"kubernetes.io/projected/7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351-kube-api-access-mkf24\") pod \"test-curl-7367-139800200791072\" (UID: \"7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351\") " pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:48.382465 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.382439    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:48.513509 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.513487    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:48.515419 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:40:48.515394    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ce8dd77_553b_4ca3_9fcb_43a0ddfd9351.slice/crio-37a61b780772327c6c7b1e3fe80e023248e32e83ca2f245afcebcc93fa5366e8 WatchSource:0}: Error finding container 37a61b780772327c6c7b1e3fe80e023248e32e83ca2f245afcebcc93fa5366e8: Status 404 returned error can't find the container with id 37a61b780772327c6c7b1e3fe80e023248e32e83ca2f245afcebcc93fa5366e8
Jul 15 20:40:48.776681 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.776604    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:48.776681 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.776629    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e51712aa83633fb2699bf31e0ac7995ee4c0e5fc5a37bc980d136d8d9a784d1"
Jul 15 20:40:48.777657 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.777631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200791072" event={"ID":"7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351","Type":"ContainerStarted","Data":"37a61b780772327c6c7b1e3fe80e023248e32e83ca2f245afcebcc93fa5366e8"}
Jul 15 20:40:48.786607 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:40:48.786575    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7367-139800200790064\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="6a2740be-f0d1-4af5-bdfd-241be8c35184" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:48.909714 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:48.909680    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj" podUID="b98cad66-4204-475b-8094-93793dcc43ad" containerName="main" containerID="cri-o://06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666" gracePeriod=45
Jul 15 20:40:49.151308 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.151285    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:40:49.170833 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:40:49.170801    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7367-139800200790064\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="6a2740be-f0d1-4af5-bdfd-241be8c35184" pod="opendatahub/test-curl-7367-139800200790064"
Jul 15 20:40:49.284040 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.283942    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98cad66-4204-475b-8094-93793dcc43ad-tls-certs\") pod \"b98cad66-4204-475b-8094-93793dcc43ad\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") "
Jul 15 20:40:49.284040 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.283988    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-tmp-dir\") pod \"b98cad66-4204-475b-8094-93793dcc43ad\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") "
Jul 15 20:40:49.284040 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.284004    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-dshm\") pod \"b98cad66-4204-475b-8094-93793dcc43ad\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") "
Jul 15 20:40:49.284040 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.284035    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2pxc\" (UniqueName: \"kubernetes.io/projected/b98cad66-4204-475b-8094-93793dcc43ad-kube-api-access-z2pxc\") pod \"b98cad66-4204-475b-8094-93793dcc43ad\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") "
Jul 15 20:40:49.284381 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.284078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-model-cache\") pod \"b98cad66-4204-475b-8094-93793dcc43ad\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") "
Jul 15 20:40:49.284381 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.284122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-home\") pod \"b98cad66-4204-475b-8094-93793dcc43ad\" (UID: \"b98cad66-4204-475b-8094-93793dcc43ad\") "
Jul 15 20:40:49.284381 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.284299    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b98cad66-4204-475b-8094-93793dcc43ad" (UID: "b98cad66-4204-475b-8094-93793dcc43ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:40:49.284508 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.284408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "b98cad66-4204-475b-8094-93793dcc43ad" (UID: "b98cad66-4204-475b-8094-93793dcc43ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:40:49.284508 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.284452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-home" (OuterVolumeSpecName: "home") pod "b98cad66-4204-475b-8094-93793dcc43ad" (UID: "b98cad66-4204-475b-8094-93793dcc43ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:40:49.286318 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.286298    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-dshm" (OuterVolumeSpecName: "dshm") pod "b98cad66-4204-475b-8094-93793dcc43ad" (UID: "b98cad66-4204-475b-8094-93793dcc43ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:40:49.286396 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.286295    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b98cad66-4204-475b-8094-93793dcc43ad-kube-api-access-z2pxc" (OuterVolumeSpecName: "kube-api-access-z2pxc") pod "b98cad66-4204-475b-8094-93793dcc43ad" (UID: "b98cad66-4204-475b-8094-93793dcc43ad"). InnerVolumeSpecName "kube-api-access-z2pxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:40:49.286396 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.286295    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98cad66-4204-475b-8094-93793dcc43ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b98cad66-4204-475b-8094-93793dcc43ad" (UID: "b98cad66-4204-475b-8094-93793dcc43ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:40:49.296555 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.296526    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a2740be-f0d1-4af5-bdfd-241be8c35184" path="/var/lib/kubelet/pods/6a2740be-f0d1-4af5-bdfd-241be8c35184/volumes"
Jul 15 20:40:49.385038 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.384998    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2pxc\" (UniqueName: \"kubernetes.io/projected/b98cad66-4204-475b-8094-93793dcc43ad-kube-api-access-z2pxc\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:49.385038 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.385027    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:49.385038 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.385037    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:49.385038 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.385046    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98cad66-4204-475b-8094-93793dcc43ad-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:49.385328 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.385054    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:49.385328 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.385063    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b98cad66-4204-475b-8094-93793dcc43ad-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:49.784016 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.783986    2563 generic.go:356] "Generic (PLEG): container finished" podID="b98cad66-4204-475b-8094-93793dcc43ad" containerID="06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666" exitCode=0
Jul 15 20:40:49.784289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.784029    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"
Jul 15 20:40:49.784289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.784025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj" event={"ID":"b98cad66-4204-475b-8094-93793dcc43ad","Type":"ContainerDied","Data":"06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666"}
Jul 15 20:40:49.784289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.784154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj" event={"ID":"b98cad66-4204-475b-8094-93793dcc43ad","Type":"ContainerDied","Data":"a89c29fcbe3e53dd909cba2814e4c0daa7d459979692458537395aa0d10a1326"}
Jul 15 20:40:49.784289 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.784179    2563 scope.go:117] "RemoveContainer" containerID="06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666"
Jul 15 20:40:49.785844 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.785822    2563 generic.go:356] "Generic (PLEG): container finished" podID="7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351" containerID="889db33ef25dea7af11f13bee6db9d634ee8cabbb40af6c99e02ceb1ea62c0dc" exitCode=0
Jul 15 20:40:49.785948 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.785875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200791072" event={"ID":"7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351","Type":"ContainerDied","Data":"889db33ef25dea7af11f13bee6db9d634ee8cabbb40af6c99e02ceb1ea62c0dc"}
Jul 15 20:40:49.792562 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.792545    2563 scope.go:117] "RemoveContainer" containerID="06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666"
Jul 15 20:40:49.792850 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:40:49.792830    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666\": container with ID starting with 06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666 not found: ID does not exist" containerID="06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666"
Jul 15 20:40:49.792917 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.792856    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666"} err="failed to get container status \"06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666\": rpc error: code = NotFound desc = could not find container \"06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666\": container with ID starting with 06f02ba4b086f6194e8eea60d47910a6ad110d71a654b9e8c6916822d374a666 not found: ID does not exist"
Jul 15 20:40:49.801161 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.801141    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"]
Jul 15 20:40:49.806438 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:49.806420    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c4ecb2/rate-test-model-c4ecb2-kserve-57f8d6dbd7-jw9fj"]
Jul 15 20:40:50.920104 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:50.920081    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:50.939536 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:50.939516    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7367-139800200791072_7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351/test-curl-7367-139800200791072/0.log"
Jul 15 20:40:50.968095 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:50.963951    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:50.970603 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:50.970580    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:50.998388 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:50.998361    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkf24\" (UniqueName: \"kubernetes.io/projected/7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351-kube-api-access-mkf24\") pod \"7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351\" (UID: \"7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351\") "
Jul 15 20:40:51.000560 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.000532    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351-kube-api-access-mkf24" (OuterVolumeSpecName: "kube-api-access-mkf24") pod "7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351" (UID: "7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351"). InnerVolumeSpecName "kube-api-access-mkf24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:40:51.087315 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.087245    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:51.087564 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.087551    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351" containerName="test-curl-7367-139800200791072"
Jul 15 20:40:51.087612 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.087566    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351" containerName="test-curl-7367-139800200791072"
Jul 15 20:40:51.087612 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.087577    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98cad66-4204-475b-8094-93793dcc43ad" containerName="main"
Jul 15 20:40:51.087612 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.087583    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98cad66-4204-475b-8094-93793dcc43ad" containerName="main"
Jul 15 20:40:51.087708 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.087640    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98cad66-4204-475b-8094-93793dcc43ad" containerName="main"
Jul 15 20:40:51.087708 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.087648    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351" containerName="test-curl-7367-139800200791072"
Jul 15 20:40:51.090413 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.090398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:51.098858 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.098832    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:51.099122 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.099100    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkf24\" (UniqueName: \"kubernetes.io/projected/7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351-kube-api-access-mkf24\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:51.200392 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.200357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nqtv\" (UniqueName: \"kubernetes.io/projected/f3da1d5c-4f28-4b88-ab75-a4e1c61f065e-kube-api-access-6nqtv\") pod \"test-curl-7367-139800200791072\" (UID: \"f3da1d5c-4f28-4b88-ab75-a4e1c61f065e\") " pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:51.296695 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.296658    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351" path="/var/lib/kubelet/pods/7ce8dd77-553b-4ca3-9fcb-43a0ddfd9351/volumes"
Jul 15 20:40:51.297095 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.297080    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b98cad66-4204-475b-8094-93793dcc43ad" path="/var/lib/kubelet/pods/b98cad66-4204-475b-8094-93793dcc43ad/volumes"
Jul 15 20:40:51.300710 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.300692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nqtv\" (UniqueName: \"kubernetes.io/projected/f3da1d5c-4f28-4b88-ab75-a4e1c61f065e-kube-api-access-6nqtv\") pod \"test-curl-7367-139800200791072\" (UID: \"f3da1d5c-4f28-4b88-ab75-a4e1c61f065e\") " pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:51.310216 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.310192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nqtv\" (UniqueName: \"kubernetes.io/projected/f3da1d5c-4f28-4b88-ab75-a4e1c61f065e-kube-api-access-6nqtv\") pod \"test-curl-7367-139800200791072\" (UID: \"f3da1d5c-4f28-4b88-ab75-a4e1c61f065e\") " pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:51.399404 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.399378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:51.526238 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.526209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:51.529597 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:40:51.529569    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3da1d5c_4f28_4b88_ab75_a4e1c61f065e.slice/crio-5ca69bc2841c1d41ee39df98fd432222376613a8de49aedb82a7ab346eddc32b WatchSource:0}: Error finding container 5ca69bc2841c1d41ee39df98fd432222376613a8de49aedb82a7ab346eddc32b: Status 404 returned error can't find the container with id 5ca69bc2841c1d41ee39df98fd432222376613a8de49aedb82a7ab346eddc32b
Jul 15 20:40:51.796741 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.796640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200791072" event={"ID":"f3da1d5c-4f28-4b88-ab75-a4e1c61f065e","Type":"ContainerStarted","Data":"5ca69bc2841c1d41ee39df98fd432222376613a8de49aedb82a7ab346eddc32b"}
Jul 15 20:40:51.797769 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.797753    2563 scope.go:117] "RemoveContainer" containerID="889db33ef25dea7af11f13bee6db9d634ee8cabbb40af6c99e02ceb1ea62c0dc"
Jul 15 20:40:51.797845 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:51.797794    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:52.803759 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:52.803700    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3da1d5c-4f28-4b88-ab75-a4e1c61f065e" containerID="ceeff261c2c81ae7e09a8a81fb7c26aa7d655640c2f951fd3d6177fcc0049d3a" exitCode=0
Jul 15 20:40:52.804190 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:52.803785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7367-139800200791072" event={"ID":"f3da1d5c-4f28-4b88-ab75-a4e1c61f065e","Type":"ContainerDied","Data":"ceeff261c2c81ae7e09a8a81fb7c26aa7d655640c2f951fd3d6177fcc0049d3a"}
Jul 15 20:40:53.937155 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:53.937134    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:53.960388 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:53.960369    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7367-139800200791072_f3da1d5c-4f28-4b88-ab75-a4e1c61f065e/test-curl-7367-139800200791072/0.log"
Jul 15 20:40:53.985665 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:53.985632    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:53.989077 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:53.989056    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7367-139800200791072"]
Jul 15 20:40:54.023840 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.023814    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nqtv\" (UniqueName: \"kubernetes.io/projected/f3da1d5c-4f28-4b88-ab75-a4e1c61f065e-kube-api-access-6nqtv\") pod \"f3da1d5c-4f28-4b88-ab75-a4e1c61f065e\" (UID: \"f3da1d5c-4f28-4b88-ab75-a4e1c61f065e\") "
Jul 15 20:40:54.025959 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.025937    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3da1d5c-4f28-4b88-ab75-a4e1c61f065e-kube-api-access-6nqtv" (OuterVolumeSpecName: "kube-api-access-6nqtv") pod "f3da1d5c-4f28-4b88-ab75-a4e1c61f065e" (UID: "f3da1d5c-4f28-4b88-ab75-a4e1c61f065e"). InnerVolumeSpecName "kube-api-access-6nqtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:40:54.125272 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.125244    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nqtv\" (UniqueName: \"kubernetes.io/projected/f3da1d5c-4f28-4b88-ab75-a4e1c61f065e-kube-api-access-6nqtv\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:54.183498 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.183469    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"]
Jul 15 20:40:54.183770 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.183758    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3da1d5c-4f28-4b88-ab75-a4e1c61f065e" containerName="test-curl-7367-139800200791072"
Jul 15 20:40:54.183821 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.183772    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3da1d5c-4f28-4b88-ab75-a4e1c61f065e" containerName="test-curl-7367-139800200791072"
Jul 15 20:40:54.183856 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.183830    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3da1d5c-4f28-4b88-ab75-a4e1c61f065e" containerName="test-curl-7367-139800200791072"
Jul 15 20:40:54.186527 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.186512    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"
Jul 15 20:40:54.188878 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.188851    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:40:54.188878 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.188872    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\""
Jul 15 20:40:54.195483 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.195452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"]
Jul 15 20:40:54.326771 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.326714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7186111e-26e8-4abb-913f-20d38e792812-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8\" (UID: \"7186111e-26e8-4abb-913f-20d38e792812\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"
Jul 15 20:40:54.427694 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.427604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7186111e-26e8-4abb-913f-20d38e792812-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8\" (UID: \"7186111e-26e8-4abb-913f-20d38e792812\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"
Jul 15 20:40:54.428268 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.428243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7186111e-26e8-4abb-913f-20d38e792812-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8\" (UID: \"7186111e-26e8-4abb-913f-20d38e792812\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"
Jul 15 20:40:54.497485 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.497450    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"
Jul 15 20:40:54.662680 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.662657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"]
Jul 15 20:40:54.665284 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:40:54.665256    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7186111e_26e8_4abb_913f_20d38e792812.slice/crio-a298cf1d0b90993b130b66aae0415148f15b21fd7a18545c8880861300bbb6e7 WatchSource:0}: Error finding container a298cf1d0b90993b130b66aae0415148f15b21fd7a18545c8880861300bbb6e7: Status 404 returned error can't find the container with id a298cf1d0b90993b130b66aae0415148f15b21fd7a18545c8880861300bbb6e7
Jul 15 20:40:54.813637 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.813602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8" event={"ID":"7186111e-26e8-4abb-913f-20d38e792812","Type":"ContainerStarted","Data":"a298cf1d0b90993b130b66aae0415148f15b21fd7a18545c8880861300bbb6e7"}
Jul 15 20:40:54.815412 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.815388    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:54.815532 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:54.815390    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ca69bc2841c1d41ee39df98fd432222376613a8de49aedb82a7ab346eddc32b"
Jul 15 20:40:54.825494 ip-10-0-141-81 kubenswrapper[2563]: E0715 20:40:54.825464    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7367-139800200791072\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="f3da1d5c-4f28-4b88-ab75-a4e1c61f065e" pod="opendatahub/test-curl-7367-139800200791072"
Jul 15 20:40:55.298150 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:55.298074    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3da1d5c-4f28-4b88-ab75-a4e1c61f065e" path="/var/lib/kubelet/pods/f3da1d5c-4f28-4b88-ab75-a4e1c61f065e/volumes"
Jul 15 20:40:55.820889 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:55.820858    2563 generic.go:356] "Generic (PLEG): container finished" podID="7186111e-26e8-4abb-913f-20d38e792812" containerID="ff51f6c713dd3912985a7d0c9cc72c1c5da6d831183c8055713d9a79e71f3f7b" exitCode=0
Jul 15 20:40:55.821065 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:55.820941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8" event={"ID":"7186111e-26e8-4abb-913f-20d38e792812","Type":"ContainerDied","Data":"ff51f6c713dd3912985a7d0c9cc72c1c5da6d831183c8055713d9a79e71f3f7b"}
Jul 15 20:40:56.950400 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:56.950376    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"
Jul 15 20:40:57.048043 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:57.048009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7186111e-26e8-4abb-913f-20d38e792812-maas-api-service-ca\") pod \"7186111e-26e8-4abb-913f-20d38e792812\" (UID: \"7186111e-26e8-4abb-913f-20d38e792812\") "
Jul 15 20:40:57.048385 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:57.048357    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7186111e-26e8-4abb-913f-20d38e792812-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7186111e-26e8-4abb-913f-20d38e792812" (UID: "7186111e-26e8-4abb-913f-20d38e792812"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:40:57.149130 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:57.149100    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7186111e-26e8-4abb-913f-20d38e792812-maas-api-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:40:57.831517 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:57.831490    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8"
Jul 15 20:40:57.831707 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:57.831490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8" event={"ID":"7186111e-26e8-4abb-913f-20d38e792812","Type":"ContainerDied","Data":"a298cf1d0b90993b130b66aae0415148f15b21fd7a18545c8880861300bbb6e7"}
Jul 15 20:40:57.831707 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:40:57.831616    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a298cf1d0b90993b130b66aae0415148f15b21fd7a18545c8880861300bbb6e7"
Jul 15 20:41:20.485579 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.485543    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"]
Jul 15 20:41:20.486084 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.485908    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7186111e-26e8-4abb-913f-20d38e792812" containerName="revoke-keys"
Jul 15 20:41:20.486084 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.485922    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7186111e-26e8-4abb-913f-20d38e792812" containerName="revoke-keys"
Jul 15 20:41:20.486084 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.485973    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7186111e-26e8-4abb-913f-20d38e792812" containerName="revoke-keys"
Jul 15 20:41:20.490292 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.490276    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"
Jul 15 20:41:20.493969 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.493950    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\""
Jul 15 20:41:20.494101 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.493955    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:41:20.502765 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.502738    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"]
Jul 15 20:41:20.643248 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.643212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2e79c2-f217-4a4c-ab57-e7027d810381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl\" (UID: \"5e2e79c2-f217-4a4c-ab57-e7027d810381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"
Jul 15 20:41:20.744187 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.744105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2e79c2-f217-4a4c-ab57-e7027d810381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl\" (UID: \"5e2e79c2-f217-4a4c-ab57-e7027d810381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"
Jul 15 20:41:20.744769 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.744750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2e79c2-f217-4a4c-ab57-e7027d810381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl\" (UID: \"5e2e79c2-f217-4a4c-ab57-e7027d810381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"
Jul 15 20:41:20.800180 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.800154    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"
Jul 15 20:41:20.941538 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:20.941512    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"]
Jul 15 20:41:20.943613 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:41:20.943585    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2e79c2_f217_4a4c_ab57_e7027d810381.slice/crio-4776f3a8016a3f0d7980d341bdc2578f88690a7a07096a763cc76ce27ac6d60d WatchSource:0}: Error finding container 4776f3a8016a3f0d7980d341bdc2578f88690a7a07096a763cc76ce27ac6d60d: Status 404 returned error can't find the container with id 4776f3a8016a3f0d7980d341bdc2578f88690a7a07096a763cc76ce27ac6d60d
Jul 15 20:41:21.942444 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:21.942409    2563 generic.go:356] "Generic (PLEG): container finished" podID="5e2e79c2-f217-4a4c-ab57-e7027d810381" containerID="f82906b4290faff95e08934e5ab206b5d2ddb79c76cdb9873d3eba64e78ef588" exitCode=0
Jul 15 20:41:21.942920 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:21.942455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl" event={"ID":"5e2e79c2-f217-4a4c-ab57-e7027d810381","Type":"ContainerDied","Data":"f82906b4290faff95e08934e5ab206b5d2ddb79c76cdb9873d3eba64e78ef588"}
Jul 15 20:41:21.942920 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:21.942492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl" event={"ID":"5e2e79c2-f217-4a4c-ab57-e7027d810381","Type":"ContainerStarted","Data":"4776f3a8016a3f0d7980d341bdc2578f88690a7a07096a763cc76ce27ac6d60d"}
Jul 15 20:41:23.077228 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:23.077204    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"
Jul 15 20:41:23.161267 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:23.161231    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2e79c2-f217-4a4c-ab57-e7027d810381-maas-api-service-ca\") pod \"5e2e79c2-f217-4a4c-ab57-e7027d810381\" (UID: \"5e2e79c2-f217-4a4c-ab57-e7027d810381\") "
Jul 15 20:41:23.161608 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:23.161581    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e2e79c2-f217-4a4c-ab57-e7027d810381-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e2e79c2-f217-4a4c-ab57-e7027d810381" (UID: "5e2e79c2-f217-4a4c-ab57-e7027d810381"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:41:23.262218 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:23.262136    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2e79c2-f217-4a4c-ab57-e7027d810381-maas-api-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Jul 15 20:41:23.952692 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:23.952656    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl"
Jul 15 20:41:23.952881 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:23.952690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl" event={"ID":"5e2e79c2-f217-4a4c-ab57-e7027d810381","Type":"ContainerDied","Data":"4776f3a8016a3f0d7980d341bdc2578f88690a7a07096a763cc76ce27ac6d60d"}
Jul 15 20:41:23.952881 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:23.952755    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4776f3a8016a3f0d7980d341bdc2578f88690a7a07096a763cc76ce27ac6d60d"
Jul 15 20:41:46.980515 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:46.980476    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56ddb87dd7-287jm_09635fa1-9c2d-4147-9906-87835043fa1b/authorino/0.log"
Jul 15 20:41:53.025829 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:53.025799    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xpzb7_9eab0a56-a6cb-43b9-a113-6a566ef1d929/manager/0.log"
Jul 15 20:41:53.269825 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:53.269797    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-x9bx2_12d9f750-efd2-446c-bc99-fb5a422a683e/copy-manifests/0.log"
Jul 15 20:41:53.278485 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:53.278422    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-x9bx2_12d9f750-efd2-446c-bc99-fb5a422a683e/manager/0.log"
Jul 15 20:41:54.310652 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:54.310621    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-514f9c-56x2k_f0438f1d-56f4-4e4a-aae1-df01352a8f90/revoke-keys/0.log"
Jul 15 20:41:54.420418 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:54.420394    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-76c4bb-7kr5h_7606859c-2d51-4d22-8cbb-c0701c922c3b/revoke-keys/0.log"
Jul 15 20:41:54.533743 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:54.533693    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-85451227-77lrg_3f5126f6-aa30-4ac1-a741-5806fd9ca28a/revoke-keys/0.log"
Jul 15 20:41:54.759048 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:54.759018    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-4d97ce-pjjpx_7e2e9dbc-0fa9-4eb7-bf42-b29e8d2d5233/revoke-keys/0.log"
Jul 15 20:41:55.113191 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:55.113164    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-b8866fb3-k9jgb_9ecdea8d-ff96-4abf-855f-02254cf22b7a/revoke-keys/0.log"
Jul 15 20:41:55.222917 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:55.222885    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-c4ecb2-czfs8_7186111e-26e8-4abb-913f-20d38e792812/revoke-keys/0.log"
Jul 15 20:41:55.332763 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:55.332716    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-8e78e5-vr6vl_5e2e79c2-f217-4a4c-ab57-e7027d810381/revoke-keys/0.log"
Jul 15 20:41:56.729811 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:56.729784    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56ddb87dd7-287jm_09635fa1-9c2d-4147-9906-87835043fa1b/authorino/0.log"
Jul 15 20:41:57.070423 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:57.070348    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8h7tn_b4705a9b-7f00-47a1-8144-d88e3b97f8ed/kuadrant-console-plugin/0.log"
Jul 15 20:41:58.155634 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:58.155607    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c4bdf9b6b-jvzrq_e6e9553d-adc7-4826-9841-c2a03eb4f771/kube-auth-proxy/0.log"
Jul 15 20:41:58.743099 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:58.743075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c74b977d-stx57_0dbd943c-6d89-4b78-8b39-ba3bb18c0e53/router/0.log"
Jul 15 20:41:59.076374 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:59.076294    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-cdhgw_a88c83dc-5f14-449e-bc75-f9fdd51537e6/main/0.log"
Jul 15 20:41:59.420236 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:59.420207    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4zcws_77c07a00-1582-47f5-8425-21b5c05817a4/main/0.log"
Jul 15 20:41:59.673254 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:41:59.673183    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-2t6fs_fb8a1abb-7071-4712-82a9-b1ba1a96ba53/main/0.log"
Jul 15 20:42:08.368501 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:08.368457    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6qndr_95419d71-9a42-48ac-a319-70c556aa50bf/global-pull-secret-syncer/0.log"
Jul 15 20:42:08.604136 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:08.604103    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xkvvl_835e5e3e-cda1-4cde-8e93-153ff2732d8a/konnectivity-agent/0.log"
Jul 15 20:42:08.683705 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:08.683626    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-81.ec2.internal_b596c108751c49733a7462cca0921090/haproxy/0.log"
Jul 15 20:42:13.454565 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:13.454532    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56ddb87dd7-287jm_09635fa1-9c2d-4147-9906-87835043fa1b/authorino/0.log"
Jul 15 20:42:13.564772 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:13.564740    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8h7tn_b4705a9b-7f00-47a1-8144-d88e3b97f8ed/kuadrant-console-plugin/0.log"
Jul 15 20:42:16.292171 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:16.292141    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwnpp_68f7322c-57e7-4676-adeb-4d2074c2a6c9/node-exporter/0.log"
Jul 15 20:42:16.318290 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:16.318262    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwnpp_68f7322c-57e7-4676-adeb-4d2074c2a6c9/kube-rbac-proxy/0.log"
Jul 15 20:42:16.362309 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:16.362283    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwnpp_68f7322c-57e7-4676-adeb-4d2074c2a6c9/init-textfile/0.log"
Jul 15 20:42:18.235821 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.235735    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"]
Jul 15 20:42:18.236269 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.236174    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2e79c2-f217-4a4c-ab57-e7027d810381" containerName="revoke-keys"
Jul 15 20:42:18.236269 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.236192    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2e79c2-f217-4a4c-ab57-e7027d810381" containerName="revoke-keys"
Jul 15 20:42:18.236269 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.236260    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e2e79c2-f217-4a4c-ab57-e7027d810381" containerName="revoke-keys"
Jul 15 20:42:18.239154 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.239133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.242284 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.242260    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mncbm\"/\"openshift-service-ca.crt\""
Jul 15 20:42:18.242920 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.242906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mncbm\"/\"kube-root-ca.crt\""
Jul 15 20:42:18.242985 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.242907    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mncbm\"/\"default-dockercfg-f7qnm\""
Jul 15 20:42:18.248945 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.248925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"]
Jul 15 20:42:18.311794 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.311761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-lib-modules\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.311973 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.311799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bktvb\" (UniqueName: \"kubernetes.io/projected/dad28bda-3e7b-4273-b6ec-734d08bed713-kube-api-access-bktvb\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.311973 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.311895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-sys\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.311973 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.311921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-podres\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.311973 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.311958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-proc\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413059 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-sys\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413059 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-podres\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413224 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-proc\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413224 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-lib-modules\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413224 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-sys\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413224 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bktvb\" (UniqueName: \"kubernetes.io/projected/dad28bda-3e7b-4273-b6ec-734d08bed713-kube-api-access-bktvb\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413409 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-podres\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413409 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-proc\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.413409 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.413324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dad28bda-3e7b-4273-b6ec-734d08bed713-lib-modules\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.421971 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.421949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bktvb\" (UniqueName: \"kubernetes.io/projected/dad28bda-3e7b-4273-b6ec-734d08bed713-kube-api-access-bktvb\") pod \"perf-node-gather-daemonset-fb2v6\" (UID: \"dad28bda-3e7b-4273-b6ec-734d08bed713\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.549866 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.549771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:18.683880 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.683852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"]
Jul 15 20:42:18.686053 ip-10-0-141-81 kubenswrapper[2563]: W0715 20:42:18.686026    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddad28bda_3e7b_4273_b6ec_734d08bed713.slice/crio-2eaf319503f28f1d2516d098bd346a24e9860af927d0884052bde6fb9801bcb4 WatchSource:0}: Error finding container 2eaf319503f28f1d2516d098bd346a24e9860af927d0884052bde6fb9801bcb4: Status 404 returned error can't find the container with id 2eaf319503f28f1d2516d098bd346a24e9860af927d0884052bde6fb9801bcb4
Jul 15 20:42:18.942912 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.942878    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/2.log"
Jul 15 20:42:18.947300 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:18.947280    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-pqfl4_05d1c048-b150-4468-a377-1390dbe7a1ad/console-operator/3.log"
Jul 15 20:42:19.208004 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:19.207919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6" event={"ID":"dad28bda-3e7b-4273-b6ec-734d08bed713","Type":"ContainerStarted","Data":"1a535d8b960f99db1038f5f3af2da88c9b25fb24587a542c18d4a8ec6f7fda7e"}
Jul 15 20:42:19.208004 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:19.207969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6" event={"ID":"dad28bda-3e7b-4273-b6ec-734d08bed713","Type":"ContainerStarted","Data":"2eaf319503f28f1d2516d098bd346a24e9860af927d0884052bde6fb9801bcb4"}
Jul 15 20:42:19.208187 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:19.208024    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:19.227629 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:19.227578    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6" podStartSLOduration=1.227563883 podStartE2EDuration="1.227563883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:42:19.226022672 +0000 UTC m=+2872.483424849" watchObservedRunningTime="2026-07-15 20:42:19.227563883 +0000 UTC m=+2872.484966092"
Jul 15 20:42:20.091014 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:20.090975    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-vdxvd_b235eecb-c48b-4c23-919d-502ca9aece5d/volume-data-source-validator/0.log"
Jul 15 20:42:21.198007 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:21.197980    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwl2l_dd40f713-e53b-4ddd-806b-298eda2be912/dns/0.log"
Jul 15 20:42:21.229020 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:21.228988    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwl2l_dd40f713-e53b-4ddd-806b-298eda2be912/kube-rbac-proxy/0.log"
Jul 15 20:42:21.435225 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:21.435195    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ljpsf_bd71fa81-6083-41ca-b02a-1b925656891b/dns-node-resolver/0.log"
Jul 15 20:42:22.111627 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:22.111599    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dhkth_e0728008-4dd7-4bb5-8220-1679c4f82a4a/node-ca/0.log"
Jul 15 20:42:23.331913 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:23.331879    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c4bdf9b6b-jvzrq_e6e9553d-adc7-4826-9841-c2a03eb4f771/kube-auth-proxy/0.log"
Jul 15 20:42:23.563756 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:23.563708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c74b977d-stx57_0dbd943c-6d89-4b78-8b39-ba3bb18c0e53/router/0.log"
Jul 15 20:42:24.336304 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:24.336274    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-874vx_42abe83f-a11e-466c-a4b5-c7a3a0c2a83f/serve-healthcheck-canary/0.log"
Jul 15 20:42:25.095751 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:25.095357    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wttqf_3f69ca9b-1411-4a49-9a5d-e533e3529026/kube-rbac-proxy/0.log"
Jul 15 20:42:25.123689 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:25.123661    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wttqf_3f69ca9b-1411-4a49-9a5d-e533e3529026/exporter/0.log"
Jul 15 20:42:25.168955 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:25.168926    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wttqf_3f69ca9b-1411-4a49-9a5d-e533e3529026/extractor/0.log"
Jul 15 20:42:25.223892 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:25.223864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-fb2v6"
Jul 15 20:42:28.094126 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:28.094088    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xpzb7_9eab0a56-a6cb-43b9-a113-6a566ef1d929/manager/0.log"
Jul 15 20:42:28.262656 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:28.262580    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-x9bx2_12d9f750-efd2-446c-bc99-fb5a422a683e/manager/0.log"
Jul 15 20:42:28.292511 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:28.292469    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-x9bx2_12d9f750-efd2-446c-bc99-fb5a422a683e/copy-manifests/0.log"
Jul 15 20:42:28.670668 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:28.670637    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c4bdf9b6b-jvzrq_e6e9553d-adc7-4826-9841-c2a03eb4f771/kube-auth-proxy/0.log"
Jul 15 20:42:28.862039 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:28.862011    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c74b977d-stx57_0dbd943c-6d89-4b78-8b39-ba3bb18c0e53/router/0.log"
Jul 15 20:42:29.640028 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:29.640003    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b4664dc78-77d8b_12adb0c6-daac-4705-9e2c-c2084f55000c/manager/0.log"
Jul 15 20:42:30.325514 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:30.325490    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c4bdf9b6b-jvzrq_e6e9553d-adc7-4826-9841-c2a03eb4f771/kube-auth-proxy/0.log"
Jul 15 20:42:30.525728 ip-10-0-141-81 kubenswrapper[2563]: I0715 20:42:30.525688    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79c74b977d-stx57_0dbd943c-6d89-4b78-8b39-ba3bb18c0e53/router/0.log"