Jul 07 02:15:55.301749 ip-10-0-142-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 02:15:55.301765 ip-10-0-142-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 02:15:55.301774 ip-10-0-142-230 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 02:15:55.302086 ip-10-0-142-230 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 02:16:05.461118 ip-10-0-142-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 02:16:05.461141 ip-10-0-142-230 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 06b5ddd4958c47a094213e908daa9be7 --
Jul 07 02:17:52.501279 ip-10-0-142-230 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 02:17:52.928994 ip-10-0-142-230 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 02:17:52.928994 ip-10-0-142-230 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 02:17:52.928994 ip-10-0-142-230 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 02:17:52.928994 ip-10-0-142-230 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 02:17:52.928994 ip-10-0-142-230 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 02:17:52.930886 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.930788    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 02:17:52.935422 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935406    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:17:52.935422 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935422    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935427    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935430    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935434    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935437    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935439    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935442    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935445    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935447    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935450    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935452    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935454    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935457    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935459    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935465    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935469    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935473    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935476    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935478    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935481    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:17:52.935489 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935484    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935487    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935489    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935492    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935495    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935498    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935501    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935504    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935506    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935508    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935512    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935515    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935517    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935520    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935522    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935525    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935529    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935533    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935535    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:17:52.935977 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935538    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935541    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935544    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935546    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935548    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935551    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935553    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935555    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935559    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935563    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935565    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935568    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935570    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935572    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935575    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935578    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935581    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935583    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935586    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:17:52.936438 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935589    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935591    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935593    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935596    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935598    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935602    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935604    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935607    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935609    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935612    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935614    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935617    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935620    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935623    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935625    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935628    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935630    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935633    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935635    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935637    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:17:52.936889 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935640    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935642    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935644    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935647    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935649    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935652    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935654    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935656    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935659    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935662    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935664    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935667    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935669    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935671    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935674    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935676    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935679    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935681    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935684    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.935686    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:17:52.937388 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936427    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936433    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936436    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936439    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936442    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936446    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936449    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936451    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936454    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936458    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936462    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936465    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936468    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936471    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936473    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936476    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936478    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936481    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936483    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:17:52.937873 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936485    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936488    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936491    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936495    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936499    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936502    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936505    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936507    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936510    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936513    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936515    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936517    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936520    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936522    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936526    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936528    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936530    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936534    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936537    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936539    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:17:52.938375 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936542    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936545    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936547    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936550    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936552    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936554    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936557    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936559    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936561    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936564    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936566    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936568    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936570    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936573    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936576    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936580    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936583    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936586    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936589    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:17:52.938854 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936591    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936594    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936597    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936599    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936602    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936605    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936607    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936610    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936612    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936614    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936617    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936620    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936622    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936625    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936628    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936630    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936632    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936635    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936637    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936640    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:17:52.939350 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936642    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936644    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936648    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936650    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936652    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936655    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936658    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936660    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936663    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936665    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936668    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936670    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936673    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936675    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936677    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936680    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936682    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936684    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936686    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936689    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:17:52.939846 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.936691    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937072    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937078    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937081    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937083    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937086    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937089    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937091    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937094    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937098    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937101    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937104    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937107    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937109    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937112    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937114    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937117    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937120    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937123    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937126    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:17:52.940341 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937129    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937131    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937134    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937136    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937138    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937141    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937143    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937146    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937148    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937150    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937153    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937155    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937157    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937159    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937162    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937165    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937167    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937169    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937176    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:17:52.940817 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937179    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937184    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937187    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937191    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937195    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937198    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937200    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937203    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937206    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937209    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937212    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937214    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937216    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937219    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937221    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937224    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937226    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937229    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937231    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937234    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:17:52.941315 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937237    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937239    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937242    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937244    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937246    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937249    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937251    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937254    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937257    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937259    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937261    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937264    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937267    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937271    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937273    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937276    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937278    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937281    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937283    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937285    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:17:52.941790 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937288    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937290    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937293    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937296    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937298    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937301    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937303    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937305    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937308    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937311    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937314    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937316    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937319    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937321    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937323    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937326    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937330    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937332    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937335    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:17:52.942503 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937338    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.937341    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937408    2585 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937415    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937421    2585 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937425    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937431    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937434    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937438    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937442    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937445    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937448    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937451    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937454    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937457    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937460    2585 flags.go:64] FLAG: --cgroup-root=""
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937463    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937466    2585 flags.go:64] FLAG: --client-ca-file=""
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937469    2585 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937472    2585 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937476    2585 flags.go:64] FLAG: --cluster-domain=""
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937479    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937482    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937485    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 02:17:52.943756 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937488    2585 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937491    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937494    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937497    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937501    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937504    2585 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937506    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937509    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937512    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937515    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937519    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937522    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937525    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937527    2585 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937532    2585 flags.go:64] FLAG: --enable-server="true"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937536    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937540    2585 flags.go:64] FLAG: --event-burst="100"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937543    2585 flags.go:64] FLAG: --event-qps="50"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937545    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937548    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937551    2585 flags.go:64] FLAG: --eviction-hard=""
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937559    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937562    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937564    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937567    2585 flags.go:64] FLAG: --eviction-soft=""
Jul 07 02:17:52.950649 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937570    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937573    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937576    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937579    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937581    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937584    2585 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937587    2585 flags.go:64] FLAG: --feature-gates=""
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937590    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937593    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937596    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937599    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937602    2585 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937605    2585 flags.go:64] FLAG: --help="false"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937607    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-142-230.ec2.internal"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937610    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937612    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937615    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937618    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937621    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937624    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937627    2585 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937631    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937634    2585 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 02:17:52.951272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937637    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937640    2585 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937643    2585 flags.go:64] FLAG: --kube-reserved=""
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937646    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937648    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937651    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937654    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937656    2585 flags.go:64] FLAG: --lock-file=""
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937660    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937663    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937666    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937671    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937673    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937676    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937679    2585 flags.go:64] FLAG: --logging-format="text"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937681    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937684    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937687    2585 flags.go:64] FLAG: --manifest-url=""
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937689    2585 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937693    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937697    2585 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937701    2585 flags.go:64] FLAG: --max-pods="110"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937703    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937706    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937709    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 02:17:52.951860 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937712    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937714    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937717    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937720    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937728    2585 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937732    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937735    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937738    2585 flags.go:64] FLAG: --pod-cidr=""
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937741    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937746    2585 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937749    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937752    2585 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937755    2585 flags.go:64] FLAG: --port="10250"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937758    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937760    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dd670c14bb9b1aee"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937770    2585 flags.go:64] FLAG: --qos-reserved=""
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937772    2585 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937775    2585 flags.go:64] FLAG: --register-node="true"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937778    2585 flags.go:64] FLAG: --register-with-taints=""
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937782    2585 flags.go:64] FLAG: --registry-burst="10"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937785    2585 flags.go:64] FLAG: --registry-qps="5"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937787    2585 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937790    2585 flags.go:64] FLAG: --reserved-memory=""
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937793    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937796    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 02:17:52.952632 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937799    2585 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937802    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937805    2585 flags.go:64] FLAG: --runonce="false"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937808    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937811    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937814    2585 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937816    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937819    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937822    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937825    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937828    2585 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937830    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937833    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937837    2585 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937840    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937842    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937846    2585 flags.go:64] FLAG: --system-cgroups=""
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937848    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937853    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937856    2585 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937859    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937864    2585 flags.go:64] FLAG: --tls-min-version=""
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937868    2585 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937870    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937873    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 02:17:52.953295 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937875    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937879    2585 flags.go:64] FLAG: --v="2"
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937883    2585 flags.go:64] FLAG: --version="false"
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937886    2585 flags.go:64] FLAG: --vmodule=""
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937890    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.937893    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938022    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938026    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938029    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938032    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938035    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938038    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938041    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938043    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938046    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938048    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938051    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938053    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938056    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938058    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:17:52.953900 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938061    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938063    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938066    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938069    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938073    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938076    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938079    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938081    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938084    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938087    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938090    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938092    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938095    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938097    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938100    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938102    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938104    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938107    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938109    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938111    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:17:52.954566 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938114    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938116    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938118    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938121    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938123    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938125    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938128    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938130    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938132    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938135    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938138    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938141    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938144    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938147    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938150    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938153    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938155    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938158    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938161    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:17:52.955320 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938163    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938167    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938172    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938174    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938176    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938179    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938181    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938184    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938186    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938189    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938191    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938194    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938196    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938198    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938201    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938203    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938205    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938208    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938210    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938212    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:17:52.955833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938215    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938217    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938220    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938222    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938224    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938227    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938229    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938232    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938234    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938236    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938239    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938242    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938244    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938247    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938250    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938253    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938255    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938258    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938260    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938262    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:17:52.956346 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938265    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938267    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938270    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938272    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938288    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.938290    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.939262    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.951998    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.952015    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.952036    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.952041    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952097    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952102    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952106    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952109    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:17:52.956833 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952112    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952114    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952116    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952119    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952123    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952126    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952129    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952132    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952135    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952138    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952140    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952143    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952146    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952148    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952150    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952153    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952155    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952158    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952160    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:17:52.957224 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952162    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952165    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952167    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952170    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952173    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952176    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952180    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952183    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952185    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952187    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952189    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952192    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952194    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952196    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952198    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952201    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952203    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952205    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952208    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952210    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:17:52.957686 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952212    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952216    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952219    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952222    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952225    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952228    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952231    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952233    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952236    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952238    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952241    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952243    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952246    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952248    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952251    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952253    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952256    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952259    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952262    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:17:52.958186 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952265    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952267    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952270    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952272    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952275    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952277    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952279    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952282    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952284    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952286    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952289    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952291    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952293    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952296    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952298    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952302    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952305    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952308    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952310    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952313    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:17:52.958653 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952315    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952317    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952320    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952322    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952325    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952327    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952329    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952332    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952334    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952336    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952339    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952342    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952344    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952347    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952349    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952352    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:17:52.959157 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952354    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.952359    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952472    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952477    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952480    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952482    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952485    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952487    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952490    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952493    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952495    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952498    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952500    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952503    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952506    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 02:17:52.959543 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952508    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952511    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952513    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952515    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952518    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952520    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952523    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952525    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952527    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952530    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952532    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952534    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952537    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952539    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952541    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952548    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952552    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952555    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952557    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952559    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 02:17:52.959902 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952562    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952564    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952566    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952568    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952571    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952573    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952575    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952578    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952580    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952582    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952585    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952587    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952590    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952592    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952594    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952596    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952599    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952601    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952603    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952607    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 02:17:52.960400 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952611    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952613    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952616    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952619    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952621    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952624    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952626    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952629    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952632    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952634    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952637    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952639    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952642    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952645    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952647    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952650    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952652    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952655    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952657    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 02:17:52.960882 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952660    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952662    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952665    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952669    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952672    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952674    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952677    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952680    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952682    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952685    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952688    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952690    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952693    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952695    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952697    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952700    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952702    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952705    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952707    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 02:17:52.961415 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952709    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952712    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952714    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952717    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952719    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952721    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952724    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:52.952726    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.952731    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.952841    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.956705    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.958128    2585 server.go:1019] "Starting client certificate rotation"
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.958230    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.958268    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 02:17:52.961865 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.960784    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 02:17:52.962232 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.961935    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 02:17:52.970826 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.970810    2585 log.go:25] "Validated CRI v1 runtime API"
Jul 07 02:17:52.974787 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.974769    2585 log.go:25] "Validated CRI v1 image API"
Jul 07 02:17:52.975671 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.975650    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 02:17:52.980401 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.980378    2585 fs.go:135] Filesystem UUIDs: map[05932a83-f65d-4110-bb59-8722b5c37e14:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b5011355-75c5-4ce8-9067-615a7c4d3272:/dev/nvme0n1p3]
Jul 07 02:17:52.980490 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.980401    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 02:17:52.985774 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.985638    2585 manager.go:216] Machine: {Timestamp:2026-07-07 02:17:52.984142286 +0000 UTC m=+0.355745921 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098338 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211574ef426df0c5de0e455d697d35 SystemUUID:ec211574-ef42-6df0-c5de-0e455d697d35 BootID:06b5ddd4-958c-47a0-9421-3e908daa9be7 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c7:cf:3b:e4:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c7:cf:3b:e4:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:6f:36:9f:55:e8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 02:17:52.985774 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.985761    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 02:17:52.985903 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.985895    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 02:17:52.988237 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.988213    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 02:17:52.988347 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.988322    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 02:17:52.988486 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.988353    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 02:17:52.989153 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.989143    2585 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 02:17:52.989184 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.989155    2585 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 02:17:52.989184 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.989164    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 02:17:52.989184 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.989178    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 02:17:52.990510 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.990499    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 02:17:52.993583 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.993571    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 02:17:52.993709 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.993699    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 02:17:52.993750 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.993734    2585 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 02:17:52.993778 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.993752    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 02:17:52.993778 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.993768    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 02:17:52.993778 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.993777    2585 kubelet.go:398] "Adding apiserver pod source"
Jul 07 02:17:52.993890 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.993790    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 02:17:52.994211 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.994199    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 02:17:52.994256 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.994217    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 02:17:52.996602 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:52.996579    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 02:17:53.000136 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.000117    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 02:17:53.000232 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.000150    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 02:17:53.001271 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001258    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001275    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001283    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001288    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001298    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001306    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001319    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001327    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001336    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001341    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001350    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 02:17:53.001351 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001355    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 02:17:53.001621 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001378    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 02:17:53.001621 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.001383    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 02:17:53.005135 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.005122    2585 server.go:1268] "Started kubelet"
Jul 07 02:17:53.005303 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.005254    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 02:17:53.005364 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.005323    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 02:17:53.005487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.005450    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 02:17:53.005813 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.005790    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 02:17:53.005813 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.005803    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 02:17:53.005978 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.005893    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 02:17:53.006265 ip-10-0-142-230 systemd[1]: Started Kubernetes Kubelet.
Jul 07 02:17:53.007020 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.007006    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 02:17:53.008163 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.008145    2585 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 02:17:53.008618 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.008593    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-88cd6"
Jul 07 02:17:53.011020 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.010054    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-230.ec2.internal.18bfe05e4836a603  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-230.ec2.internal,UID:ip-10-0-142-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-230.ec2.internal,},FirstTimestamp:2026-07-07 02:17:53.005094403 +0000 UTC m=+0.376698038,LastTimestamp:2026-07-07 02:17:53.005094403 +0000 UTC m=+0.376698038,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-230.ec2.internal,}"
Jul 07 02:17:53.011164 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.011116    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 02:17:53.012129 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012109    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 02:17:53.012184 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012122    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 02:17:53.012242 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.012182    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.012667 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012599    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 02:17:53.012667 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012600    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 02:17:53.012667 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012638    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 02:17:53.012814 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012753    2585 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 02:17:53.012814 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012763    2585 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 02:17:53.013008 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.012988    2585 factory.go:55] Registering systemd factory
Jul 07 02:17:53.013008 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.013010    2585 factory.go:223] Registration of the systemd container factory successfully
Jul 07 02:17:53.013292 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.013208    2585 factory.go:153] Registering CRI-O factory
Jul 07 02:17:53.013292 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.013218    2585 factory.go:223] Registration of the crio container factory successfully
Jul 07 02:17:53.013292 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.013264    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 02:17:53.013292 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.013290    2585 factory.go:103] Registering Raw factory
Jul 07 02:17:53.013451 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.013310    2585 manager.go:1196] Started watching for new ooms in manager
Jul 07 02:17:53.013788 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.013772    2585 manager.go:349] Starting recovery of all containers
Jul 07 02:17:53.016415 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.016386    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-88cd6"
Jul 07 02:17:53.024752 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.024560    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:17:53.026211 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.026176    2585 manager.go:354] Recovery completed
Jul 07 02:17:53.031951 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.031917    2585 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-230.ec2.internal\" not found" node="ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.032007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.031978    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:17:53.034021 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.034007    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:17:53.034142 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.034034    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:17:53.034142 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.034044    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:17:53.034650 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.034637    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 02:17:53.034650 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.034649    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 02:17:53.034734 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.034666    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 02:17:53.034734 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.034684    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:17:53.035645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.035629    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:17:53.035700 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.035661    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:17:53.035700 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.035676    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:17:53.035762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.035749    2585 policy_none.go:49] "None policy: Start"
Jul 07 02:17:53.036828 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.036817    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 02:17:53.036859 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.036836    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 02:17:53.038089 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.038078    2585 policy_none.go:47] "Start"
Jul 07 02:17:53.076577 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.076551    2585 manager.go:337] "Starting Device Plugin manager"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.076585    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.076595    2585 server.go:87] "Starting device plugin registration server"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.076892    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.076941    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.077017    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.077093    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.077102    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.077763    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 02:17:53.083964 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.077817    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.146599 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.146553    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 02:17:53.147946 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.147912    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 02:17:53.148085 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.147966    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 02:17:53.148085 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.147993    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 02:17:53.148085 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.148031    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 02:17:53.150375 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.150356    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:17:53.177274 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.177236    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:17:53.178381 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.178361    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:17:53.178492 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.178399    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:17:53.178492 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.178415    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:17:53.178492 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.178446    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.184526 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.184478    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.184526 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.184501    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-230.ec2.internal\": node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.202234 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.202209    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.249016 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.248981    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"]
Jul 07 02:17:53.249165 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.249084    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:17:53.250391 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.250373    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:17:53.250487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.250409    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:17:53.250487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.250424    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:17:53.252003 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.251992    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:17:53.252131 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252115    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.252175 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252149    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:17:53.252790 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252773    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:17:53.252790 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252787    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:17:53.252912 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252806    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:17:53.252912 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252808    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:17:53.252912 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252830    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:17:53.252912 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.252820    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:17:53.254005 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.253991    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.254069 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.254021    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 02:17:53.254757 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.254744    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 02:17:53.254821 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.254766    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 02:17:53.254821 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.254782    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID"
Jul 07 02:17:53.271534 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.271509    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-230.ec2.internal\" not found" node="ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.274775 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.274758    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-230.ec2.internal\" not found" node="ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.302524 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.302496    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.314902 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.314873    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d502e0a41f1530561a0d77d959ab94c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"4d502e0a41f1530561a0d77d959ab94c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.315004 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.314904    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d502e0a41f1530561a0d77d959ab94c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"4d502e0a41f1530561a0d77d959ab94c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.315004 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.314922    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61aad07ba0bedc3a4e130d11147f6dd3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"61aad07ba0bedc3a4e130d11147f6dd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.403529 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.403489    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.415721 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.415692    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d502e0a41f1530561a0d77d959ab94c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"4d502e0a41f1530561a0d77d959ab94c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.415721 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.415724    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d502e0a41f1530561a0d77d959ab94c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"4d502e0a41f1530561a0d77d959ab94c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.415863 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.415739    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61aad07ba0bedc3a4e130d11147f6dd3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"61aad07ba0bedc3a4e130d11147f6dd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.415863 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.415787    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d502e0a41f1530561a0d77d959ab94c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"4d502e0a41f1530561a0d77d959ab94c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.415863 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.415788    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d502e0a41f1530561a0d77d959ab94c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"4d502e0a41f1530561a0d77d959ab94c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.415863 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.415832    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61aad07ba0bedc3a4e130d11147f6dd3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"61aad07ba0bedc3a4e130d11147f6dd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.503607 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.503554    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.573987 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.573957    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.578764 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.578309    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.603707 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.603668    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.704209 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.704176    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.804780 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:53.804708    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found"
Jul 07 02:17:53.854596 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.854566    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:17:53.912629 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.912603    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.923453 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.923424    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 02:17:53.925311 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.925298    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"
Jul 07 02:17:53.933150 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.933135    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 02:17:53.957347 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.957321    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 02:17:53.957490 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.957467    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 02:17:53.957554 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.957475    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 02:17:53.957554 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.957478    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 02:17:53.994669 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:53.994640    2585 apiserver.go:52] "Watching apiserver"
Jul 07 02:17:54.012434 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.012410    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 02:17:54.015758 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.015739    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 02:17:54.016119 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.016099    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-lklr2","openshift-ovn-kubernetes/ovnkube-node-s4lcd","kube-system/konnectivity-agent-rkw64","openshift-cluster-node-tuning-operator/tuned-bt8ls","openshift-dns/node-resolver-qbffq","openshift-image-registry/node-ca-gkgb6","openshift-multus/multus-24qgz","openshift-multus/multus-additional-cni-plugins-s6z8n","openshift-network-diagnostics/network-check-target-bfq5k","openshift-network-operator/iptables-alerter-v4vlx","kube-system/global-pull-secret-syncer-n82t2","kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"]
Jul 07 02:17:54.017576 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.017563    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.018840 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.018817    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.019949 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.019914    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:54.020332 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.020319    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 02:17:54.020921 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.020897    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 02:17:54.021205 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.021143    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 02:12:54 +0000 UTC" deadline="2028-01-03 16:52:27.973481757 +0000 UTC"
Jul 07 02:17:54.021287 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.021211    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13094h34m33.952278828s"
Jul 07 02:17:54.021287 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.021240    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 02:17:54.021653 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.021631    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 02:17:54.021748 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.021725    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7l7f9\""
Jul 07 02:17:54.022895 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.022121    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 02:17:54.022895 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.022265    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.022895 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.022578    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-68wjn\""
Jul 07 02:17:54.023100 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.023040    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 02:17:54.023100 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.023044    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 02:17:54.023371 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.023355    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 02:17:54.023611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.023550    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 02:17:54.023835 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.023808    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.025007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.024985    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-z5n9w\""
Jul 07 02:17:54.025180 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.025162    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:17:54.025293 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.025178    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 02:17:54.025374 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.025323    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8mlzf\""
Jul 07 02:17:54.025551 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.025538    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 02:17:54.026343 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.026327    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.027578 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.027560    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:54.027672 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.027609    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 02:17:54.027672 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.027651    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:17:54.027750 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.027659    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4m52t\""
Jul 07 02:17:54.028037 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.028021    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 02:17:54.028712 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.028693    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:54.028794 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.028775    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:17:54.029015 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.028989    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 02:17:54.029015 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.028997    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 02:17:54.029270 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.029240    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7l6j8\""
Jul 07 02:17:54.029350 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.029277    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 02:17:54.030235 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.030216    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.031596 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.031581    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.033006 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.032990    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.033147 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.033125    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:17:54.034352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034335    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.034568 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034549    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 02:17:54.034639 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034582    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 02:17:54.034639 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034551    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-45rcd\""
Jul 07 02:17:54.034639 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034633    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 02:17:54.034762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034698    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 02:17:54.034762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034733    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wtl6t\""
Jul 07 02:17:54.034762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034735    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 02:17:54.034762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034745    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 02:17:54.034947 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034775    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 02:17:54.034947 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.034888    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 02:17:54.035638 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.035626    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 02:17:54.037318 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.037303    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 02:17:54.037367 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.037303    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 02:17:54.037588 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.037575    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tqzxv\""
Jul 07 02:17:54.051079 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.051058    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ndbgg"
Jul 07 02:17:54.060178 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.060125    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ndbgg"
Jul 07 02:17:54.115869 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.114794    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 02:17:54.118337 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118304    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff0a071f-968b-4626-80e8-c5483ff06d14-host-slash\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.118448 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118341    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n8qx\" (UniqueName: \"kubernetes.io/projected/ff0a071f-968b-4626-80e8-c5483ff06d14-kube-api-access-8n8qx\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.118448 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118362    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-systemd\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.118448 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118394    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-var-lib-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.118448 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118418    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-os-release\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.118448 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118443    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.118645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118466    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-kubelet\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.118645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118487    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-cni-netd\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.118645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118508    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-system-cni-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.118645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118544    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-etc-kubernetes\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.118645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118571    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t52jk\" (UniqueName: \"kubernetes.io/projected/c980fc15-667f-435a-8215-11d614462009-kube-api-access-t52jk\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:54.118645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118594    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.118645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118638    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-log-socket\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118668    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118693    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118706    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-os-release\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118732    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-cni-bin\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118758    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/38bb8a4b-467c-44c3-b2f0-01095131796a-konnectivity-ca\") pod \"konnectivity-agent-rkw64\" (UID: \"38bb8a4b-467c-44c3-b2f0-01095131796a\") " pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118783    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-ovn\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118802    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzkd4\" (UniqueName: \"kubernetes.io/projected/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-kube-api-access-hzkd4\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118827    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-cnibin\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.118877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118850    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118880    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-daemon-config\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118896    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6b7p\" (UniqueName: \"kubernetes.io/projected/bdba4aa3-a5bb-49f2-8dab-424b741e708a-kube-api-access-t6b7p\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118916    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-lib-modules\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118953    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-var-lib-kubelet\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.118995    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-ovnkube-config\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119025    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-hosts-file\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119044    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-sys-fs\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119064    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-tuned\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119080    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/478e4abc-993a-4b3f-96c9-8e78d6401ccf-tmp\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25ftz\" (UniqueName: \"kubernetes.io/projected/478e4abc-993a-4b3f-96c9-8e78d6401ccf-kube-api-access-25ftz\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119121    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-run-ovn-kubernetes\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119141    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/278a6bcd-e480-4cab-9a83-a53245336588-host\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-device-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.119220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119196    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119229    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119248    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/38bb8a4b-467c-44c3-b2f0-01095131796a-agent-certs\") pod \"konnectivity-agent-rkw64\" (UID: \"38bb8a4b-467c-44c3-b2f0-01095131796a\") " pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119267    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k9md\" (UniqueName: \"kubernetes.io/projected/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-kube-api-access-8k9md\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119282    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/278a6bcd-e480-4cab-9a83-a53245336588-serviceca\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119295    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119310    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-netns\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119338    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-hostroot\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119370    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119397    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/25e0d312-fbb7-40bd-93b3-ce1dd410426f-kubelet-config\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119430    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/25e0d312-fbb7-40bd-93b3-ce1dd410426f-dbus\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119452    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119475    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-modprobe-d\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119500    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-socket-dir-parent\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119542    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-kubelet\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119589    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-conf-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119621    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-tmp-dir\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.119768 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119644    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdba4aa3-a5bb-49f2-8dab-424b741e708a-cni-binary-copy\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119679    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-etc-selinux\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119712    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwsjz\" (UniqueName: \"kubernetes.io/projected/15c4030e-e654-4f98-b5ec-51fccf9ac20e-kube-api-access-lwsjz\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119755    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5mwf\" (UniqueName: \"kubernetes.io/projected/fae1d72b-9030-4479-a63c-77d5339e0e22-kube-api-access-z5mwf\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119788    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-k8s-cni-cncf-io\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119836    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff0a071f-968b-4626-80e8-c5483ff06d14-iptables-alerter-script\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119862    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-socket-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119894    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-kubernetes\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119916    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-run-netns\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119956    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-node-log\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.119992    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-cni-bin\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120019    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-ovnkube-script-lib\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120040    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cnibin\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120078    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysconfig\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120115    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120119    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysctl-d\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120168    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-run\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.120428 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120203    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-env-overrides\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120226    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-cni-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120246    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-systemd\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120284    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-sys\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120311    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-multus-certs\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120334    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvgwt\" (UniqueName: \"kubernetes.io/projected/278a6bcd-e480-4cab-9a83-a53245336588-kube-api-access-lvgwt\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-registration-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120380    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysctl-conf\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120400    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-host\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120422    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-systemd-units\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120443    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-slash\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120464    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-etc-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120487    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fae1d72b-9030-4479-a63c-77d5339e0e22-ovn-node-metrics-cert\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120511    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-system-cni-dir\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.121074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.120539    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-cni-multus\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.189923 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:54.189882    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d502e0a41f1530561a0d77d959ab94c.slice/crio-3bc146c67455876457ea7b125e62456008dbfca8450894136f4916bc1391b8bd WatchSource:0}: Error finding container 3bc146c67455876457ea7b125e62456008dbfca8450894136f4916bc1391b8bd: Status 404 returned error can't find the container with id 3bc146c67455876457ea7b125e62456008dbfca8450894136f4916bc1391b8bd
Jul 07 02:17:54.190569 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:54.190543    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61aad07ba0bedc3a4e130d11147f6dd3.slice/crio-8c65218b4d48f65832af2cc6416e4807fe458681f7edc85aa080b8406b40afff WatchSource:0}: Error finding container 8c65218b4d48f65832af2cc6416e4807fe458681f7edc85aa080b8406b40afff: Status 404 returned error can't find the container with id 8c65218b4d48f65832af2cc6416e4807fe458681f7edc85aa080b8406b40afff
Jul 07 02:17:54.197092 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.197072    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:17:54.221194 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221163    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.221194 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221194    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/38bb8a4b-467c-44c3-b2f0-01095131796a-agent-certs\") pod \"konnectivity-agent-rkw64\" (UID: \"38bb8a4b-467c-44c3-b2f0-01095131796a\") " pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:54.221410 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221218    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k9md\" (UniqueName: \"kubernetes.io/projected/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-kube-api-access-8k9md\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.221410 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221242    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/278a6bcd-e480-4cab-9a83-a53245336588-serviceca\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.221410 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221257    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.221410 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221276    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.221410 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221305    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-netns\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.221410 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221358    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-netns\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.221410 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221399    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221423    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-hostroot\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221522    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221552    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-hostroot\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221583    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/25e0d312-fbb7-40bd-93b3-ce1dd410426f-kubelet-config\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221610    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/25e0d312-fbb7-40bd-93b3-ce1dd410426f-dbus\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221633    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.221653    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221657    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-modprobe-d\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221684    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-socket-dir-parent\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221703    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.221736    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs podName:c980fc15-667f-435a-8215-11d614462009 nodeName:}" failed. No retries permitted until 2026-07-07 02:17:54.721692062 +0000 UTC m=+2.093295684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs") pod "network-metrics-daemon-lklr2" (UID: "c980fc15-667f-435a-8215-11d614462009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:54.221754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221754    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-socket-dir-parent\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221769    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-kubelet\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221797    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-conf-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221777    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221820    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/25e0d312-fbb7-40bd-93b3-ce1dd410426f-dbus\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221847    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-kubelet\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221859    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-conf-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221863    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-modprobe-d\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221822    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-tmp-dir\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221884    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/278a6bcd-e480-4cab-9a83-a53245336588-serviceca\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221909    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdba4aa3-a5bb-49f2-8dab-424b741e708a-cni-binary-copy\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221956    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-etc-selinux\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.221984    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwsjz\" (UniqueName: \"kubernetes.io/projected/15c4030e-e654-4f98-b5ec-51fccf9ac20e-kube-api-access-lwsjz\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222010    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5mwf\" (UniqueName: \"kubernetes.io/projected/fae1d72b-9030-4479-a63c-77d5339e0e22-kube-api-access-z5mwf\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222062    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-k8s-cni-cncf-io\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222110    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-etc-selinux\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222118    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-k8s-cni-cncf-io\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222166    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-tmp-dir\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.222305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222229    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff0a071f-968b-4626-80e8-c5483ff06d14-iptables-alerter-script\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222271    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-socket-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222307    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-kubernetes\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222366    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-run-netns\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222373    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-socket-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222392    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-kubernetes\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222390    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-node-log\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222413    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdba4aa3-a5bb-49f2-8dab-424b741e708a-cni-binary-copy\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222423    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-cni-bin\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222427    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-run-netns\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222456    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-ovnkube-script-lib\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222471    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-node-log\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222487    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cnibin\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222492    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-cni-bin\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222516    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysconfig\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222540    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysctl-d\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222541    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cnibin\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-run\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222585    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-env-overrides\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222600    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-cni-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222600    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysconfig\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222616    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-systemd\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-sys\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222661    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-multus-certs\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222667    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-run\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222685    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvgwt\" (UniqueName: \"kubernetes.io/projected/278a6bcd-e480-4cab-9a83-a53245336588-kube-api-access-lvgwt\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222710    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-registration-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222716    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff0a071f-968b-4626-80e8-c5483ff06d14-iptables-alerter-script\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222722    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysctl-d\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222735    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysctl-conf\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222761    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-host\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222772    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-cni-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222783    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-systemd-units\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222789    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-sys\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222720    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-systemd\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222843    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-registration-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.223966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222826    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-slash\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222853    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/25e0d312-fbb7-40bd-93b3-ce1dd410426f-kubelet-config\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222888    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-etc-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222905    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-host\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222905    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-systemd-units\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222916    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fae1d72b-9030-4479-a63c-77d5339e0e22-ovn-node-metrics-cert\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222967    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-run-multus-certs\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.222993    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-system-cni-dir\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223007    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-etc-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223040    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-sysctl-conf\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223047    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-slash\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223054    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-ovnkube-script-lib\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223074    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-cni-multus\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223091    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-system-cni-dir\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223103    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff0a071f-968b-4626-80e8-c5483ff06d14-host-slash\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223138    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-cni-multus\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223143    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n8qx\" (UniqueName: \"kubernetes.io/projected/ff0a071f-968b-4626-80e8-c5483ff06d14-kube-api-access-8n8qx\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.224777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223168    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-systemd\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223184    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff0a071f-968b-4626-80e8-c5483ff06d14-host-slash\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223193    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-var-lib-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223215    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-var-lib-openvswitch\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223221    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-os-release\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223530    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-env-overrides\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223641    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223677    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-systemd\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223687    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-os-release\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223725    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-kubelet\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223765    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-cni-netd\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223789    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-system-cni-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223815    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-kubelet\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223828    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-cni-netd\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223841    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-system-cni-dir\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223861    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-etc-kubernetes\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223897    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-etc-kubernetes\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.225538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223910    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t52jk\" (UniqueName: \"kubernetes.io/projected/c980fc15-667f-435a-8215-11d614462009-kube-api-access-t52jk\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223956    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.223990    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-log-socket\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224019    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224043    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224084    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-os-release\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.224096    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224108    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-cni-bin\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224135    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/38bb8a4b-467c-44c3-b2f0-01095131796a-konnectivity-ca\") pod \"konnectivity-agent-rkw64\" (UID: \"38bb8a4b-467c-44c3-b2f0-01095131796a\") " pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.224155    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret podName:25e0d312-fbb7-40bd-93b3-ce1dd410426f nodeName:}" failed. No retries permitted until 2026-07-07 02:17:54.724138894 +0000 UTC m=+2.095742535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret") pod "global-pull-secret-syncer-n82t2" (UID: "25e0d312-fbb7-40bd-93b3-ce1dd410426f") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224187    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-ovn\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224210    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224232    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzkd4\" (UniqueName: \"kubernetes.io/projected/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-kube-api-access-hzkd4\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224272    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-cnibin\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224277    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-run-ovn\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224310    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224310    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.226050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224324    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-host-var-lib-cni-bin\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224355    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-daemon-config\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224382    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6b7p\" (UniqueName: \"kubernetes.io/projected/bdba4aa3-a5bb-49f2-8dab-424b741e708a-kube-api-access-t6b7p\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224405    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-lib-modules\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224431    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-var-lib-kubelet\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224457    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-ovnkube-config\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224482    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-hosts-file\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224515    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-sys-fs\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224539    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-tuned\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224561    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-cnibin\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224563    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/478e4abc-993a-4b3f-96c9-8e78d6401ccf-tmp\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224592    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224607    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25ftz\" (UniqueName: \"kubernetes.io/projected/478e4abc-993a-4b3f-96c9-8e78d6401ccf-kube-api-access-25ftz\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224637    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-run-ovn-kubernetes\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224649    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-var-lib-kubelet\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224663    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/278a6bcd-e480-4cab-9a83-a53245336588-host\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224688    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-device-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.226570 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224699    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/38bb8a4b-467c-44c3-b2f0-01095131796a-konnectivity-ca\") pod \"konnectivity-agent-rkw64\" (UID: \"38bb8a4b-467c-44c3-b2f0-01095131796a\") " pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224715    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224774    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdba4aa3-a5bb-49f2-8dab-424b741e708a-os-release\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224829    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224886    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-host-run-ovn-kubernetes\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.224968    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/278a6bcd-e480-4cab-9a83-a53245336588-host\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225017    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-device-dir\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225059    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-hosts-file\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225093    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/478e4abc-993a-4b3f-96c9-8e78d6401ccf-lib-modules\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225149    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15c4030e-e654-4f98-b5ec-51fccf9ac20e-sys-fs\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225334    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bdba4aa3-a5bb-49f2-8dab-424b741e708a-multus-daemon-config\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225422    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fae1d72b-9030-4479-a63c-77d5339e0e22-log-socket\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225759    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fae1d72b-9030-4479-a63c-77d5339e0e22-ovn-node-metrics-cert\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.225806    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/38bb8a4b-467c-44c3-b2f0-01095131796a-agent-certs\") pod \"konnectivity-agent-rkw64\" (UID: \"38bb8a4b-467c-44c3-b2f0-01095131796a\") " pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.226187    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fae1d72b-9030-4479-a63c-77d5339e0e22-ovnkube-config\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.226725    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/478e4abc-993a-4b3f-96c9-8e78d6401ccf-tmp\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.227123 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.227012    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/478e4abc-993a-4b3f-96c9-8e78d6401ccf-etc-tuned\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.233673 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.233650    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k9md\" (UniqueName: \"kubernetes.io/projected/255f2d58-2da4-4f7c-8e74-e2b5d583f2a2-kube-api-access-8k9md\") pod \"node-resolver-qbffq\" (UID: \"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2\") " pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:54.235919 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.235874    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:17:54.235919 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.235901    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:17:54.235919 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.235915    2585 projected.go:196] Error preparing data for projected volume kube-api-access-l9szf for pod openshift-network-diagnostics/network-check-target-bfq5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:54.236242 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:54.235993    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf podName:8adfce1b-afab-453b-a0ad-97c3e0dbdf6a nodeName:}" failed. No retries permitted until 2026-07-07 02:17:54.735974424 +0000 UTC m=+2.107578063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l9szf" (UniqueName: "kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf") pod "network-check-target-bfq5k" (UID: "8adfce1b-afab-453b-a0ad-97c3e0dbdf6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:54.237096 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.237069    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n8qx\" (UniqueName: \"kubernetes.io/projected/ff0a071f-968b-4626-80e8-c5483ff06d14-kube-api-access-8n8qx\") pod \"iptables-alerter-v4vlx\" (UID: \"ff0a071f-968b-4626-80e8-c5483ff06d14\") " pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:54.237322 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.237272    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5mwf\" (UniqueName: \"kubernetes.io/projected/fae1d72b-9030-4479-a63c-77d5339e0e22-kube-api-access-z5mwf\") pod \"ovnkube-node-s4lcd\" (UID: \"fae1d72b-9030-4479-a63c-77d5339e0e22\") " pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:54.237368 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.237337    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwsjz\" (UniqueName: \"kubernetes.io/projected/15c4030e-e654-4f98-b5ec-51fccf9ac20e-kube-api-access-lwsjz\") pod \"aws-ebs-csi-driver-node-bw2wx\" (UID: \"15c4030e-e654-4f98-b5ec-51fccf9ac20e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:54.237368 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.237301    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t52jk\" (UniqueName: \"kubernetes.io/projected/c980fc15-667f-435a-8215-11d614462009-kube-api-access-t52jk\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:54.237741 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.237722    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25ftz\" (UniqueName: \"kubernetes.io/projected/478e4abc-993a-4b3f-96c9-8e78d6401ccf-kube-api-access-25ftz\") pod \"tuned-bt8ls\" (UID: \"478e4abc-993a-4b3f-96c9-8e78d6401ccf\") " pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:54.237885 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.237867    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzkd4\" (UniqueName: \"kubernetes.io/projected/e0050f4f-59a5-4548-a62d-cb7c61cff8d1-kube-api-access-hzkd4\") pod \"multus-additional-cni-plugins-s6z8n\" (UID: \"e0050f4f-59a5-4548-a62d-cb7c61cff8d1\") " pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:54.238165 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.238143    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6b7p\" (UniqueName: \"kubernetes.io/projected/bdba4aa3-a5bb-49f2-8dab-424b741e708a-kube-api-access-t6b7p\") pod \"multus-24qgz\" (UID: \"bdba4aa3-a5bb-49f2-8dab-424b741e708a\") " pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.238231 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.238218    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvgwt\" (UniqueName: \"kubernetes.io/projected/278a6bcd-e480-4cab-9a83-a53245336588-kube-api-access-lvgwt\") pod \"node-ca-gkgb6\" (UID: \"278a6bcd-e480-4cab-9a83-a53245336588\") " pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:54.352166 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:54.352081    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-24qgz"
Jul 07 02:17:54.361015 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:54.360986    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdba4aa3_a5bb_49f2_8dab_424b741e708a.slice/crio-24c6ff060c0363a4f03aa8aa98b14b02cd8f0825dff9cdaaef2553844d1eeb80 WatchSource:0}: Error finding container 24c6ff060c0363a4f03aa8aa98b14b02cd8f0825dff9cdaaef2553844d1eeb80: Status 404 returned error can't find the container with id 24c6ff060c0363a4f03aa8aa98b14b02cd8f0825dff9cdaaef2553844d1eeb80
Jul 07 02:17:55.620613 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.620590    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx"
Jul 07 02:17:55.627810 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.627773    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c4030e_e654_4f98_b5ec_51fccf9ac20e.slice/crio-bfe16bb87c1ee2c84bbea753c8a68f99977e849ee9a62f0c9c8c28440788247e WatchSource:0}: Error finding container bfe16bb87c1ee2c84bbea753c8a68f99977e849ee9a62f0c9c8c28440788247e: Status 404 returned error can't find the container with id bfe16bb87c1ee2c84bbea753c8a68f99977e849ee9a62f0c9c8c28440788247e
Jul 07 02:17:55.640792 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.640760    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:17:55.648947 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.648922    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bb8a4b_467c_44c3_b2f0_01095131796a.slice/crio-7b11a76812268f81e5ed9a2c84bb428db70424fbdf9fce6713e7e52621fcf11a WatchSource:0}: Error finding container 7b11a76812268f81e5ed9a2c84bb428db70424fbdf9fce6713e7e52621fcf11a: Status 404 returned error can't find the container with id 7b11a76812268f81e5ed9a2c84bb428db70424fbdf9fce6713e7e52621fcf11a
Jul 07 02:17:55.656038 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.656019    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bt8ls"
Jul 07 02:17:55.664496 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.664473    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478e4abc_993a_4b3f_96c9_8e78d6401ccf.slice/crio-9995fdbb271574aac44e4ccf933fc8876b2b302d3a1f1383295380e0ad950d7f WatchSource:0}: Error finding container 9995fdbb271574aac44e4ccf933fc8876b2b302d3a1f1383295380e0ad950d7f: Status 404 returned error can't find the container with id 9995fdbb271574aac44e4ccf933fc8876b2b302d3a1f1383295380e0ad950d7f
Jul 07 02:17:55.669654 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.669637    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qbffq"
Jul 07 02:17:55.675982 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.675965    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gkgb6"
Jul 07 02:17:55.676484 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.676462    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255f2d58_2da4_4f7c_8e74_e2b5d583f2a2.slice/crio-38f7840fb0b60f3af06304393d6bac9aeff104149cba498c533a25530a146231 WatchSource:0}: Error finding container 38f7840fb0b60f3af06304393d6bac9aeff104149cba498c533a25530a146231: Status 404 returned error can't find the container with id 38f7840fb0b60f3af06304393d6bac9aeff104149cba498c533a25530a146231
Jul 07 02:17:55.683347 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.683156    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod278a6bcd_e480_4cab_9a83_a53245336588.slice/crio-e552237f3249b147f6789bae7e5b855c780f2954094aec6f14226c0f25578621 WatchSource:0}: Error finding container e552237f3249b147f6789bae7e5b855c780f2954094aec6f14226c0f25578621: Status 404 returned error can't find the container with id e552237f3249b147f6789bae7e5b855c780f2954094aec6f14226c0f25578621
Jul 07 02:17:55.691066 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.691039    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:17:55.698650 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.698630    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae1d72b_9030_4479_a63c_77d5339e0e22.slice/crio-0986a2e148ea02c4e23caa1d8ad83458b426b361acccf37b0e4614b1539f33e3 WatchSource:0}: Error finding container 0986a2e148ea02c4e23caa1d8ad83458b426b361acccf37b0e4614b1539f33e3: Status 404 returned error can't find the container with id 0986a2e148ea02c4e23caa1d8ad83458b426b361acccf37b0e4614b1539f33e3
Jul 07 02:17:55.705883 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.705863    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v4vlx"
Jul 07 02:17:55.710871 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.710849    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s6z8n"
Jul 07 02:17:55.713099 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.713079    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0a071f_968b_4626_80e8_c5483ff06d14.slice/crio-6794f5255dd23944cb61f4201be29b28758011a0420416cac05bd0d2007f34af WatchSource:0}: Error finding container 6794f5255dd23944cb61f4201be29b28758011a0420416cac05bd0d2007f34af: Status 404 returned error can't find the container with id 6794f5255dd23944cb61f4201be29b28758011a0420416cac05bd0d2007f34af
Jul 07 02:17:55.718533 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:17:55.718512    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0050f4f_59a5_4548_a62d_cb7c61cff8d1.slice/crio-df5c9a087a33e8bec2bee38df59cda4febb87252931a9940d25485ac0ea19528 WatchSource:0}: Error finding container df5c9a087a33e8bec2bee38df59cda4febb87252931a9940d25485ac0ea19528: Status 404 returned error can't find the container with id df5c9a087a33e8bec2bee38df59cda4febb87252931a9940d25485ac0ea19528
Jul 07 02:17:55.972976 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.972940    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:55.973137 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:55.972988    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:55.973137 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:55.973004    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:55.973137 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:55.973057    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret podName:25e0d312-fbb7-40bd-93b3-ce1dd410426f nodeName:}" failed. No retries permitted until 2026-07-07 02:17:56.973037173 +0000 UTC m=+3.101123856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret") pod "global-pull-secret-syncer-n82t2" (UID: "25e0d312-fbb7-40bd-93b3-ce1dd410426f") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:55.973264 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:55.973135    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:55.973264 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:55.973192    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs podName:c980fc15-667f-435a-8215-11d614462009 nodeName:}" failed. No retries permitted until 2026-07-07 02:17:56.973177441 +0000 UTC m=+3.101264115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs") pod "network-metrics-daemon-lklr2" (UID: "c980fc15-667f-435a-8215-11d614462009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:56.073377 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.073324    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:56.073590 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.073572    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:17:56.073665 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.073597    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:17:56.073665 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.073610    2585 projected.go:196] Error preparing data for projected volume kube-api-access-l9szf for pod openshift-network-diagnostics/network-check-target-bfq5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:56.073758 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.073667    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf podName:8adfce1b-afab-453b-a0ad-97c3e0dbdf6a nodeName:}" failed. No retries permitted until 2026-07-07 02:17:57.073648638 +0000 UTC m=+3.201735321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9szf" (UniqueName: "kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf") pod "network-check-target-bfq5k" (UID: "8adfce1b-afab-453b-a0ad-97c3e0dbdf6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:56.306011 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.305898    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 02:12:55 +0000 UTC" deadline="2028-03-14 19:36:59.61135101 +0000 UTC"
Jul 07 02:17:56.306011 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.305963    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14801h19m3.305392826s"
Jul 07 02:17:56.391851 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.391820    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:56.392021 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.391952    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:17:56.414497 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.414416    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bt8ls" event={"ID":"478e4abc-993a-4b3f-96c9-8e78d6401ccf","Type":"ContainerStarted","Data":"9995fdbb271574aac44e4ccf933fc8876b2b302d3a1f1383295380e0ad950d7f"}
Jul 07 02:17:56.430644 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.430575    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx" event={"ID":"15c4030e-e654-4f98-b5ec-51fccf9ac20e","Type":"ContainerStarted","Data":"bfe16bb87c1ee2c84bbea753c8a68f99977e849ee9a62f0c9c8c28440788247e"}
Jul 07 02:17:56.444669 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.444621    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qbffq" event={"ID":"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2","Type":"ContainerStarted","Data":"38f7840fb0b60f3af06304393d6bac9aeff104149cba498c533a25530a146231"}
Jul 07 02:17:56.450005 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.449976    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:17:56.471813 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.471771    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rkw64" event={"ID":"38bb8a4b-467c-44c3-b2f0-01095131796a","Type":"ContainerStarted","Data":"7b11a76812268f81e5ed9a2c84bb428db70424fbdf9fce6713e7e52621fcf11a"}
Jul 07 02:17:56.486298 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.485149    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24qgz" event={"ID":"bdba4aa3-a5bb-49f2-8dab-424b741e708a","Type":"ContainerStarted","Data":"24c6ff060c0363a4f03aa8aa98b14b02cd8f0825dff9cdaaef2553844d1eeb80"}
Jul 07 02:17:56.495055 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.495020    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"4d502e0a41f1530561a0d77d959ab94c","Type":"ContainerStarted","Data":"3bc146c67455876457ea7b125e62456008dbfca8450894136f4916bc1391b8bd"}
Jul 07 02:17:56.500181 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.500145    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" event={"ID":"61aad07ba0bedc3a4e130d11147f6dd3","Type":"ContainerStarted","Data":"8c65218b4d48f65832af2cc6416e4807fe458681f7edc85aa080b8406b40afff"}
Jul 07 02:17:56.512052 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.512021    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerStarted","Data":"df5c9a087a33e8bec2bee38df59cda4febb87252931a9940d25485ac0ea19528"}
Jul 07 02:17:56.515470 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.515322    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v4vlx" event={"ID":"ff0a071f-968b-4626-80e8-c5483ff06d14","Type":"ContainerStarted","Data":"6794f5255dd23944cb61f4201be29b28758011a0420416cac05bd0d2007f34af"}
Jul 07 02:17:56.521282 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.521252    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"0986a2e148ea02c4e23caa1d8ad83458b426b361acccf37b0e4614b1539f33e3"}
Jul 07 02:17:56.530725 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.530664    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gkgb6" event={"ID":"278a6bcd-e480-4cab-9a83-a53245336588","Type":"ContainerStarted","Data":"e552237f3249b147f6789bae7e5b855c780f2954094aec6f14226c0f25578621"}
Jul 07 02:17:56.714624 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.713458    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:17:56.982833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.982734    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:56.983007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:56.982844    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:56.983007 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.982980    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:56.983121 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.983045    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret podName:25e0d312-fbb7-40bd-93b3-ce1dd410426f nodeName:}" failed. No retries permitted until 2026-07-07 02:17:58.98302539 +0000 UTC m=+5.111112071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret") pod "global-pull-secret-syncer-n82t2" (UID: "25e0d312-fbb7-40bd-93b3-ce1dd410426f") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:56.983121 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.983118    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:56.983241 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:56.983156    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs podName:c980fc15-667f-435a-8215-11d614462009 nodeName:}" failed. No retries permitted until 2026-07-07 02:17:58.983142874 +0000 UTC m=+5.111229552 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs") pod "network-metrics-daemon-lklr2" (UID: "c980fc15-667f-435a-8215-11d614462009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:57.083571 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:57.083532    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:57.083833 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:57.083811    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:17:57.083916 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:57.083840    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:17:57.083916 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:57.083853    2585 projected.go:196] Error preparing data for projected volume kube-api-access-l9szf for pod openshift-network-diagnostics/network-check-target-bfq5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:57.083916 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:57.083913    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf podName:8adfce1b-afab-453b-a0ad-97c3e0dbdf6a nodeName:}" failed. No retries permitted until 2026-07-07 02:17:59.083894194 +0000 UTC m=+5.211980879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9szf" (UniqueName: "kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf") pod "network-check-target-bfq5k" (UID: "8adfce1b-afab-453b-a0ad-97c3e0dbdf6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:57.307280 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:57.307119    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 02:12:55 +0000 UTC" deadline="2028-04-04 00:47:42.79740129 +0000 UTC"
Jul 07 02:17:57.307280 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:57.307194    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15286h29m45.490212017s"
Jul 07 02:17:57.363111 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:57.362865    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 02:17:57.393118 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:57.392362    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:57.393118 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:57.392514    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:17:57.393118 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:57.392942    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:57.393118 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:57.393062    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:17:58.391845 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:58.391810    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:58.392336 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:58.391950    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:17:59.002664 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:59.002616    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:59.002845 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:59.002704    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:17:59.002845 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.002834    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:59.002975 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.002901    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret podName:25e0d312-fbb7-40bd-93b3-ce1dd410426f nodeName:}" failed. No retries permitted until 2026-07-07 02:18:03.00287992 +0000 UTC m=+9.130966597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret") pod "global-pull-secret-syncer-n82t2" (UID: "25e0d312-fbb7-40bd-93b3-ce1dd410426f") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:17:59.002975 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.002967    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:59.003088 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.003006    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs podName:c980fc15-667f-435a-8215-11d614462009 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:03.002992361 +0000 UTC m=+9.131079038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs") pod "network-metrics-daemon-lklr2" (UID: "c980fc15-667f-435a-8215-11d614462009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:17:59.103907 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:59.103650    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:59.103907 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.103837    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:17:59.103907 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.103859    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:17:59.103907 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.103873    2585 projected.go:196] Error preparing data for projected volume kube-api-access-l9szf for pod openshift-network-diagnostics/network-check-target-bfq5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:59.104245 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.103934    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf podName:8adfce1b-afab-453b-a0ad-97c3e0dbdf6a nodeName:}" failed. No retries permitted until 2026-07-07 02:18:03.103914159 +0000 UTC m=+9.232000835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9szf" (UniqueName: "kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf") pod "network-check-target-bfq5k" (UID: "8adfce1b-afab-453b-a0ad-97c3e0dbdf6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:17:59.391948 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:59.391870    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:17:59.392384 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.391989    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:17:59.392474 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:17:59.392397    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:17:59.392563 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:17:59.392538    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:00.392477 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:00.391724    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:00.392477 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:00.391860    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:01.392274 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:01.392238    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:01.392494 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:01.392383    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:01.392494 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:01.392459    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:01.392914 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:01.392537    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:02.391761 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:02.391722    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:02.391950 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:02.391855    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:03.037013 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:03.036974    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:03.037374 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:03.037052    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:03.037374 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.037203    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:18:03.037374 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.037262    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs podName:c980fc15-667f-435a-8215-11d614462009 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:11.03724457 +0000 UTC m=+17.165331244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs") pod "network-metrics-daemon-lklr2" (UID: "c980fc15-667f-435a-8215-11d614462009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:18:03.037707 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.037668    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:18:03.037826 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.037722    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret podName:25e0d312-fbb7-40bd-93b3-ce1dd410426f nodeName:}" failed. No retries permitted until 2026-07-07 02:18:11.037706801 +0000 UTC m=+17.165793473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret") pod "global-pull-secret-syncer-n82t2" (UID: "25e0d312-fbb7-40bd-93b3-ce1dd410426f") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:18:03.138388 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:03.138335    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:03.138585 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.138568    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:18:03.138662 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.138595    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:18:03.138662 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.138609    2585 projected.go:196] Error preparing data for projected volume kube-api-access-l9szf for pod openshift-network-diagnostics/network-check-target-bfq5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:18:03.138771 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.138668    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf podName:8adfce1b-afab-453b-a0ad-97c3e0dbdf6a nodeName:}" failed. No retries permitted until 2026-07-07 02:18:11.138650468 +0000 UTC m=+17.266737142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9szf" (UniqueName: "kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf") pod "network-check-target-bfq5k" (UID: "8adfce1b-afab-453b-a0ad-97c3e0dbdf6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:18:03.393157 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:03.392527    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:03.393157 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.392657    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:03.393157 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:03.393039    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:03.393157 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:03.393109    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:04.395663 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:04.395146    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:04.395663 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:04.395330    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:05.392381 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:05.392356    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:05.392536 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:05.392512    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:05.392651 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:05.392360    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:05.392755 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:05.392732    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:06.392253 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:06.392214    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:06.392722 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:06.392351    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:07.391872 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:07.391825    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:07.392048 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:07.391956    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:07.392048 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:07.392015    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:07.392161 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:07.392133    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:08.392171 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:08.392136    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:08.392678 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:08.392249    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:09.391805 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:09.391767    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:09.391805 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:09.391821    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:09.392093 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:09.391910    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:09.392093 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:09.392060    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:10.391797 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:10.391766    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:10.392246 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:10.391885    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:11.096498 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:11.096174    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:11.096498 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:11.096279    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:11.096498 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.096341    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:18:11.096498 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.096396    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:18:11.096498 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.096420    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs podName:c980fc15-667f-435a-8215-11d614462009 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:27.096397656 +0000 UTC m=+33.224484347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs") pod "network-metrics-daemon-lklr2" (UID: "c980fc15-667f-435a-8215-11d614462009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:18:11.096498 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.096460    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret podName:25e0d312-fbb7-40bd-93b3-ce1dd410426f nodeName:}" failed. No retries permitted until 2026-07-07 02:18:27.09642851 +0000 UTC m=+33.224515179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret") pod "global-pull-secret-syncer-n82t2" (UID: "25e0d312-fbb7-40bd-93b3-ce1dd410426f") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:18:11.196799 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:11.196759    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:11.196969 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.196939    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:18:11.196969 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.196966    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:18:11.197069 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.196976    2585 projected.go:196] Error preparing data for projected volume kube-api-access-l9szf for pod openshift-network-diagnostics/network-check-target-bfq5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:18:11.197069 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.197030    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf podName:8adfce1b-afab-453b-a0ad-97c3e0dbdf6a nodeName:}" failed. No retries permitted until 2026-07-07 02:18:27.197012143 +0000 UTC m=+33.325098816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9szf" (UniqueName: "kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf") pod "network-check-target-bfq5k" (UID: "8adfce1b-afab-453b-a0ad-97c3e0dbdf6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:18:11.392644 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:11.392565    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:11.393016 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:11.392577    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:11.393016 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.392720    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:11.393016 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:11.392779    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:12.392005 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:12.391967    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:12.392230 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:12.392095    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:13.392528 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:13.392488    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:13.392940 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:13.392488    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:13.392940 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:13.392626    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:13.392940 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:13.392712    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:14.393458 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:14.393405    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:14.393892 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:14.393549    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:15.392499 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:15.392459    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:15.392691 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:15.392634    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:15.393242 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:15.392777    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:15.393242 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:15.392854    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:16.392863 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:16.392689    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:16.393236 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:16.392945    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:16.593685 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:16.593598    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bt8ls" event={"ID":"478e4abc-993a-4b3f-96c9-8e78d6401ccf","Type":"ContainerStarted","Data":"a065df22b57f05e7b73d8e2e747c842603f34c0f9c2b6248bb73fb7cc8ef98aa"}
Jul 07 02:18:16.595186 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:16.595159    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-24qgz" event={"ID":"bdba4aa3-a5bb-49f2-8dab-424b741e708a","Type":"ContainerStarted","Data":"556feaf02716d2348ee151b2faaf8c809f5a2fd72b1695226be76e48d9af5ba0"}
Jul 07 02:18:16.601563 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:16.601539    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" event={"ID":"61aad07ba0bedc3a4e130d11147f6dd3","Type":"ContainerStarted","Data":"1a3dccf2a1087eb6e67fe9f7190f8a1b550bceba9d4bf7f6cd151ac0f2c28ac5"}
Jul 07 02:18:16.610931 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:16.610887    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bt8ls" podStartSLOduration=2.254265751 podStartE2EDuration="22.610874492s" totalImagesPullingTime="20.356608741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:16.610504801 +0000 UTC m=+22.738591491" watchObservedRunningTime="2026-07-07 02:18:16.610874492 +0000 UTC m=+22.738961182"
Jul 07 02:18:16.642167 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:16.642116    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-24qgz" podStartSLOduration=2.204373922 podStartE2EDuration="22.642099815s" totalImagesPullingTime="20.437725893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:16.641717489 +0000 UTC m=+22.769804181" watchObservedRunningTime="2026-07-07 02:18:16.642099815 +0000 UTC m=+22.770186506"
Jul 07 02:18:16.642418 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:16.642396    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" podStartSLOduration=21.642387665 podStartE2EDuration="21.642387665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:16.624372361 +0000 UTC m=+22.752459053" watchObservedRunningTime="2026-07-07 02:18:16.642387665 +0000 UTC m=+22.770474356"
Jul 07 02:18:17.392738 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.392509    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:17.392883 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.392538    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:17.392883 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:17.392789    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:17.392883 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:17.392837    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:17.605771 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.605743    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerStarted","Data":"c48e5d2f5195ec0fbd117b11457c247bb51c98dfe599bc2247aea2400b5e8860"}
Jul 07 02:18:17.608674 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.608643    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v4vlx" event={"ID":"ff0a071f-968b-4626-80e8-c5483ff06d14","Type":"ContainerStarted","Data":"4171c8198315d45d131e651290da2a8c2e3af435278afeff22673de86792ac59"}
Jul 07 02:18:17.613341 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.613317    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:18:17.613660 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.613640    2585 generic.go:356] "Generic (PLEG): container finished" podID="fae1d72b-9030-4479-a63c-77d5339e0e22" containerID="5a979c271f858acfa0ebe3d0884960b9807eeb7c9686f445d3664b1812e882bc" exitCode=1
Jul 07 02:18:17.613739 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.613725    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"19fb81c425006676c7658230237cdfced5476f9d9bc26ef127747607abf02a9f"}
Jul 07 02:18:17.613816 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.613799    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"cc95df439bc2a86b5972b428888772b8e5e57387365a4e24ea97f0f0a43ce0ba"}
Jul 07 02:18:17.613875 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.613857    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"5fbc5e1ad65a803a1c1a7828c2648dd31ac12ac332c4ea698118f2cceea1fc60"}
Jul 07 02:18:17.613929 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.613905    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"7d706ba00bb69ee44119a56f4c41f76d3285b123a46670c15300e336634e5c6c"}
Jul 07 02:18:17.613982 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.613953    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerDied","Data":"5a979c271f858acfa0ebe3d0884960b9807eeb7c9686f445d3664b1812e882bc"}
Jul 07 02:18:17.614033 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.614006    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"f96c786afbf69cf586634c137e03a897b4a3d841d91fbfdc5655aae752aee9ad"}
Jul 07 02:18:17.615164 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.615143    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gkgb6" event={"ID":"278a6bcd-e480-4cab-9a83-a53245336588","Type":"ContainerStarted","Data":"040173757eaf2bcd51ad0f3dd690952862f556a1d2301de2e2296253630aa489"}
Jul 07 02:18:17.616875 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.616838    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx" event={"ID":"15c4030e-e654-4f98-b5ec-51fccf9ac20e","Type":"ContainerStarted","Data":"7c784effb8f5d9c637d837a69335d0afad7faa423e3d780121fa68323dec62ce"}
Jul 07 02:18:17.618493 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.618472    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qbffq" event={"ID":"255f2d58-2da4-4f7c-8e74-e2b5d583f2a2","Type":"ContainerStarted","Data":"545e47343d7f0db69a722a75288b44d5ff9459bb66e5090e32b8482914ca69cc"}
Jul 07 02:18:17.620352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.620170    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rkw64" event={"ID":"38bb8a4b-467c-44c3-b2f0-01095131796a","Type":"ContainerStarted","Data":"98acda5a6a5c9c30a65ae097fdcda86072364ee3d1ec2dfa2e4e253cfcff3aae"}
Jul 07 02:18:17.621588 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.621538    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"4d502e0a41f1530561a0d77d959ab94c","Type":"ContainerStarted","Data":"9816607a501ebf6cd28b6a8a32ee16ef38f3718627a79be94744a554a082898c"}
Jul 07 02:18:17.664833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.664737    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rkw64" podStartSLOduration=3.305885719 podStartE2EDuration="23.664723298s" totalImagesPullingTime="20.358837579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:17.664510913 +0000 UTC m=+23.792597604" watchObservedRunningTime="2026-07-07 02:18:17.664723298 +0000 UTC m=+23.792809988"
Jul 07 02:18:17.681510 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.681461    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qbffq" podStartSLOduration=3.333542752 podStartE2EDuration="23.68142772s" totalImagesPullingTime="20.347884968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:17.680761501 +0000 UTC m=+23.808848183" watchObservedRunningTime="2026-07-07 02:18:17.68142772 +0000 UTC m=+23.809514408"
Jul 07 02:18:17.697674 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:17.695264    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gkgb6" podStartSLOduration=3.382080209 podStartE2EDuration="23.695237891s" totalImagesPullingTime="20.313157682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:17.694837942 +0000 UTC m=+23.822924624" watchObservedRunningTime="2026-07-07 02:18:17.695237891 +0000 UTC m=+23.823324580"
Jul 07 02:18:18.392339 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:18.392301    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:18.392603 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:18.392462    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:18.624310 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:18.624273    2585 generic.go:356] "Generic (PLEG): container finished" podID="e0050f4f-59a5-4548-a62d-cb7c61cff8d1" containerID="c48e5d2f5195ec0fbd117b11457c247bb51c98dfe599bc2247aea2400b5e8860" exitCode=0
Jul 07 02:18:18.624783 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:18.624358    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerDied","Data":"c48e5d2f5195ec0fbd117b11457c247bb51c98dfe599bc2247aea2400b5e8860"}
Jul 07 02:18:18.625870 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:18.625851    2585 generic.go:356] "Generic (PLEG): container finished" podID="4d502e0a41f1530561a0d77d959ab94c" containerID="9816607a501ebf6cd28b6a8a32ee16ef38f3718627a79be94744a554a082898c" exitCode=0
Jul 07 02:18:18.625946 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:18.625924    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"4d502e0a41f1530561a0d77d959ab94c","Type":"ContainerDied","Data":"9816607a501ebf6cd28b6a8a32ee16ef38f3718627a79be94744a554a082898c"}
Jul 07 02:18:18.655436 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:18.655352    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v4vlx" podStartSLOduration=4.349922711 podStartE2EDuration="24.65533638s" totalImagesPullingTime="20.305413669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:17.708650005 +0000 UTC m=+23.836736697" watchObservedRunningTime="2026-07-07 02:18:18.65533638 +0000 UTC m=+24.783423070"
Jul 07 02:18:18.812049 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:18.812000    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 02:18:19.334783 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.333765    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T02:18:18.812018884Z","UUID":"2f2c53f4-4f94-4e3f-ab77-34da6f2e6a0c","Handler":null,"Name":"","Endpoint":""}
Jul 07 02:18:19.337308 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.336124    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 02:18:19.337308 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.336169    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 02:18:19.392272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.392249    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:19.392401 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.392287    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:19.392401 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:19.392375    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:19.392567 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:19.392538    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:19.644206 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.643915    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx" event={"ID":"15c4030e-e654-4f98-b5ec-51fccf9ac20e","Type":"ContainerStarted","Data":"e9092119c8fe78ea3a2b89df81fc8bac4792c655a679af00ac8b9999cfa83744"}
Jul 07 02:18:19.646020 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.645983    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"4d502e0a41f1530561a0d77d959ab94c","Type":"ContainerStarted","Data":"04f210a8c6bcb24691374b2c87bcdb7299f94a0b8e69b7e0b4ac46fdcff16a48"}
Jul 07 02:18:19.650083 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.650039    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:18:19.650438 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.650414    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"05667342bf03409e8dc35ed44a1266c754ababb053a1301a45e4861e300a8934"}
Jul 07 02:18:19.665391 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:19.665330    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" podStartSLOduration=24.665310679 podStartE2EDuration="24.665310679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:19.664674268 +0000 UTC m=+25.792760962" watchObservedRunningTime="2026-07-07 02:18:19.665310679 +0000 UTC m=+25.793397371"
Jul 07 02:18:20.392186 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:20.392149    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:20.392378 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:20.392294    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:20.655032 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:20.654951    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx" event={"ID":"15c4030e-e654-4f98-b5ec-51fccf9ac20e","Type":"ContainerStarted","Data":"eaa1dd97fe996ea2cab20b4ab8a3cb5da5408ea521775ab44cb32a20cd6d59eb"}
Jul 07 02:18:20.674027 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:20.673982    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw2wx" podStartSLOduration=4.099596395 podStartE2EDuration="26.673959661s" totalImagesPullingTime="22.574363266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:20.672350348 +0000 UTC m=+26.800437042" watchObservedRunningTime="2026-07-07 02:18:20.673959661 +0000 UTC m=+26.802046355"
Jul 07 02:18:20.982225 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:20.982195    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:18:20.982913 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:20.982889    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:18:21.392773 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:21.392698    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:21.392916 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:21.392817    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:21.392916 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:21.392876    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:21.393035 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:21.392991    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:21.657848 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:21.657767    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:18:21.658328 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:21.658310    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rkw64"
Jul 07 02:18:22.392370 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:22.392341    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:22.392571 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:22.392487    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:23.392670 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:23.392479    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:23.393140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:23.392518    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:23.393140 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:23.392765    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:23.393140 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:23.392820    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:23.664199 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:23.664128    2585 generic.go:356] "Generic (PLEG): container finished" podID="e0050f4f-59a5-4548-a62d-cb7c61cff8d1" containerID="0552465087db50c4ec7513b2381e68925425412a7b6b39f85ea7b3d4e009f985" exitCode=0
Jul 07 02:18:23.664372 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:23.664209    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerDied","Data":"0552465087db50c4ec7513b2381e68925425412a7b6b39f85ea7b3d4e009f985"}
Jul 07 02:18:23.667743 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:23.667727    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:18:23.668088 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:23.668055    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"337c1f74d49b3139c632361bbea0acc922b2dce8497aa4995b6ebbc469c84c6c"}
Jul 07 02:18:23.668501 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:23.668486    2585 scope.go:117] "RemoveContainer" containerID="5a979c271f858acfa0ebe3d0884960b9807eeb7c9686f445d3664b1812e882bc"
Jul 07 02:18:24.394646 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.394609    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:24.395268 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:24.394758    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:24.673854 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.673833    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:18:24.674244 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.674220    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" event={"ID":"fae1d72b-9030-4479-a63c-77d5339e0e22","Type":"ContainerStarted","Data":"f5b3a1e9a22501bed3efb0db3acbe17b46f7d60709cf71fab25e9176989cb391"}
Jul 07 02:18:24.674597 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.674565    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:18:24.674702 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.674644    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:18:24.674702 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.674689    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:18:24.693293 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.693269    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:18:24.693607 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.693582    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:18:24.715373 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:24.715318    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd" podStartSLOduration=9.728200170000001 podStartE2EDuration="30.715301117s" totalImagesPullingTime="20.987100947s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:24.714816001 +0000 UTC m=+30.842902701" watchObservedRunningTime="2026-07-07 02:18:24.715301117 +0000 UTC m=+30.843387808"
Jul 07 02:18:25.018911 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.018695    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bfq5k"]
Jul 07 02:18:25.019138 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.019016    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:25.019138 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:25.019113    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:25.019326 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.019305    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n82t2"]
Jul 07 02:18:25.019399 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.019389    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:25.019503 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:25.019485    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:25.021475 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.021432    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lklr2"]
Jul 07 02:18:25.021623 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.021558    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:25.021679 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:25.021660    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:25.678386 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.678338    2585 generic.go:356] "Generic (PLEG): container finished" podID="e0050f4f-59a5-4548-a62d-cb7c61cff8d1" containerID="dc17a2c18c7003e94398c8c3511c1e2294fbfe915fb56902790c2c81f9fa0789" exitCode=0
Jul 07 02:18:25.678782 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:25.678415    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerDied","Data":"dc17a2c18c7003e94398c8c3511c1e2294fbfe915fb56902790c2c81f9fa0789"}
Jul 07 02:18:26.392554 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:26.392519    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:26.392736 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:26.392532    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:26.392736 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:26.392655    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:26.392736 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:26.392690    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:26.683318 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:26.683280    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerStarted","Data":"ecae105b323e2cd45526c270fc3ef627aadf6aa1ccd8ec7e454f4c09a3c0cbec"}
Jul 07 02:18:27.118131 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:27.118092    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:27.118290 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:27.118158    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:27.118290 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.118245    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 02:18:27.118290 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.118258    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:18:27.118433 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.118317    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs podName:c980fc15-667f-435a-8215-11d614462009 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:59.118298676 +0000 UTC m=+65.246385345 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs") pod "network-metrics-daemon-lklr2" (UID: "c980fc15-667f-435a-8215-11d614462009") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 02:18:27.118433 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.118335    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret podName:25e0d312-fbb7-40bd-93b3-ce1dd410426f nodeName:}" failed. No retries permitted until 2026-07-07 02:18:59.11832695 +0000 UTC m=+65.246413623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret") pod "global-pull-secret-syncer-n82t2" (UID: "25e0d312-fbb7-40bd-93b3-ce1dd410426f") : object "kube-system"/"original-pull-secret" not registered
Jul 07 02:18:27.219367 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:27.219332    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:27.219554 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.219512    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 02:18:27.219554 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.219531    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 02:18:27.219554 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.219541    2585 projected.go:196] Error preparing data for projected volume kube-api-access-l9szf for pod openshift-network-diagnostics/network-check-target-bfq5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:18:27.219651 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.219593    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf podName:8adfce1b-afab-453b-a0ad-97c3e0dbdf6a nodeName:}" failed. No retries permitted until 2026-07-07 02:18:59.219578453 +0000 UTC m=+65.347665122 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9szf" (UniqueName: "kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf") pod "network-check-target-bfq5k" (UID: "8adfce1b-afab-453b-a0ad-97c3e0dbdf6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 02:18:27.392762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:27.392681    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:27.392899 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:27.392796    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lklr2" podUID="c980fc15-667f-435a-8215-11d614462009"
Jul 07 02:18:27.687843 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:27.687811    2585 generic.go:356] "Generic (PLEG): container finished" podID="e0050f4f-59a5-4548-a62d-cb7c61cff8d1" containerID="ecae105b323e2cd45526c270fc3ef627aadf6aa1ccd8ec7e454f4c09a3c0cbec" exitCode=0
Jul 07 02:18:27.688198 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:27.687871    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerDied","Data":"ecae105b323e2cd45526c270fc3ef627aadf6aa1ccd8ec7e454f4c09a3c0cbec"}
Jul 07 02:18:28.395229 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:28.395201    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:28.395409 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:28.395313    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bfq5k" podUID="8adfce1b-afab-453b-a0ad-97c3e0dbdf6a"
Jul 07 02:18:28.395409 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:28.395350    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:28.395517 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:28.395416    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n82t2" podUID="25e0d312-fbb7-40bd-93b3-ce1dd410426f"
Jul 07 02:18:29.190578 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.190549    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeReady"
Jul 07 02:18:29.191015 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.190725    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 02:18:29.228414 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.228382    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77f64d5f4b-qxgph"]
Jul 07 02:18:29.230459 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.230419    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.235147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.235125    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 02:18:29.235300 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.235280    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-56fvm\""
Jul 07 02:18:29.235379 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.235328    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 02:18:29.235603 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.235587    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 02:18:29.244059 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.244035    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 02:18:29.245494 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.245471    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kh65r"]
Jul 07 02:18:29.247519 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.247496    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xqnxc"]
Jul 07 02:18:29.248487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.248469    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.249360 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.249341    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77f64d5f4b-qxgph"]
Jul 07 02:18:29.249480 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.249415    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.251490 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.251468    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tp7rb\""
Jul 07 02:18:29.251750 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.251731    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 02:18:29.252059 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.252035    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 02:18:29.252273 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.252252    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 02:18:29.252384 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.252368    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 02:18:29.252515 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.252500    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 02:18:29.253216 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.253197    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4v9ss\""
Jul 07 02:18:29.255335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.255318    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kh65r"]
Jul 07 02:18:29.262591 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.262529    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xqnxc"]
Jul 07 02:18:29.333961 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.333880    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-bound-sa-token\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.333961 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.333911    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgp4b\" (UniqueName: \"kubernetes.io/projected/5244c428-40dd-4db7-acb9-5723befa306b-kube-api-access-tgp4b\") pod \"ingress-canary-kh65r\" (UID: \"5244c428-40dd-4db7-acb9-5723befa306b\") " pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.333961 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.333945    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqtqb\" (UniqueName: \"kubernetes.io/projected/ca58ca66-f969-4f25-85ff-b95d5ac9595b-kube-api-access-dqtqb\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.333964    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-installation-pull-secrets\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334003    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334052    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca58ca66-f969-4f25-85ff-b95d5ac9595b-config-volume\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334066    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334080    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-ca-trust-extracted\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-trusted-ca\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334126    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npkfs\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-kube-api-access-npkfs\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334182    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca58ca66-f969-4f25-85ff-b95d5ac9595b-tmp-dir\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.334227 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334211    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-image-registry-private-configuration\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.334784 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334235    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-certificates\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.334784 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.334278    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5244c428-40dd-4db7-acb9-5723befa306b-cert\") pod \"ingress-canary-kh65r\" (UID: \"5244c428-40dd-4db7-acb9-5723befa306b\") " pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.392363 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.392313    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:29.395620 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.395596    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 02:18:29.395620 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.395607    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kthkt\""
Jul 07 02:18:29.434912 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.434886    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5244c428-40dd-4db7-acb9-5723befa306b-cert\") pod \"ingress-canary-kh65r\" (UID: \"5244c428-40dd-4db7-acb9-5723befa306b\") " pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.434912 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.434916    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-bound-sa-token\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.434935    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgp4b\" (UniqueName: \"kubernetes.io/projected/5244c428-40dd-4db7-acb9-5723befa306b-kube-api-access-tgp4b\") pod \"ingress-canary-kh65r\" (UID: \"5244c428-40dd-4db7-acb9-5723befa306b\") " pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.434960    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtqb\" (UniqueName: \"kubernetes.io/projected/ca58ca66-f969-4f25-85ff-b95d5ac9595b-kube-api-access-dqtqb\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.434978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-installation-pull-secrets\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.434999    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435035    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca58ca66-f969-4f25-85ff-b95d5ac9595b-config-volume\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435056    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435073    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-ca-trust-extracted\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435099    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-trusted-ca\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435126    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npkfs\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-kube-api-access-npkfs\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435157    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca58ca66-f969-4f25-85ff-b95d5ac9595b-tmp-dir\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.435607 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435182    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-image-registry-private-configuration\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435607 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435205    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-certificates\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.435607 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.435207    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:18:29.435607 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.435317    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls podName:ca58ca66-f969-4f25-85ff-b95d5ac9595b nodeName:}" failed. No retries permitted until 2026-07-07 02:18:29.935297168 +0000 UTC m=+36.063383839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls") pod "dns-default-xqnxc" (UID: "ca58ca66-f969-4f25-85ff-b95d5ac9595b") : secret "dns-default-metrics-tls" not found
Jul 07 02:18:29.435787 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.435766    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-ca-trust-extracted\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.436058 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.436034    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca58ca66-f969-4f25-85ff-b95d5ac9595b-config-volume\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.436403 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.436375    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-certificates\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.436773 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.436637    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-trusted-ca\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.436849 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.436654    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca58ca66-f969-4f25-85ff-b95d5ac9595b-tmp-dir\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.436849 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.435220    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:18:29.436849 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.436821    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f64d5f4b-qxgph: secret "image-registry-tls" not found
Jul 07 02:18:29.436999 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.436924    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls podName:ef8961d7-c2fb-4066-ad2b-feb9b153a2b8 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:29.936896136 +0000 UTC m=+36.064982819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls") pod "image-registry-77f64d5f4b-qxgph" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8") : secret "image-registry-tls" not found
Jul 07 02:18:29.440208 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.440180    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-image-registry-private-configuration\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.440349 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.440328    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5244c428-40dd-4db7-acb9-5723befa306b-cert\") pod \"ingress-canary-kh65r\" (UID: \"5244c428-40dd-4db7-acb9-5723befa306b\") " pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.441698 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.441673    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-installation-pull-secrets\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.444767 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.444736    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-bound-sa-token\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.446052 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.446020    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npkfs\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-kube-api-access-npkfs\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.446140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.446054    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgp4b\" (UniqueName: \"kubernetes.io/projected/5244c428-40dd-4db7-acb9-5723befa306b-kube-api-access-tgp4b\") pod \"ingress-canary-kh65r\" (UID: \"5244c428-40dd-4db7-acb9-5723befa306b\") " pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.447896 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.447872    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqtqb\" (UniqueName: \"kubernetes.io/projected/ca58ca66-f969-4f25-85ff-b95d5ac9595b-kube-api-access-dqtqb\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.561893 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.561859    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kh65r"
Jul 07 02:18:29.724762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.724719    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kh65r"]
Jul 07 02:18:29.729128 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:29.729092    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5244c428_40dd_4db7_acb9_5723befa306b.slice/crio-ff47ec224484f325bfeee6c389873e278ffa48f9f6d03442e65d47e0c5ad4d48 WatchSource:0}: Error finding container ff47ec224484f325bfeee6c389873e278ffa48f9f6d03442e65d47e0c5ad4d48: Status 404 returned error can't find the container with id ff47ec224484f325bfeee6c389873e278ffa48f9f6d03442e65d47e0c5ad4d48
Jul 07 02:18:29.940583 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.940547    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:29.940762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:29.940610    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:29.940762 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.940721    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:18:29.940762 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.940731    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:18:29.940762 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.940749    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f64d5f4b-qxgph: secret "image-registry-tls" not found
Jul 07 02:18:29.940918 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.940800    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls podName:ca58ca66-f969-4f25-85ff-b95d5ac9595b nodeName:}" failed. No retries permitted until 2026-07-07 02:18:30.940774453 +0000 UTC m=+37.068861127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls") pod "dns-default-xqnxc" (UID: "ca58ca66-f969-4f25-85ff-b95d5ac9595b") : secret "dns-default-metrics-tls" not found
Jul 07 02:18:29.940918 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:29.940818    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls podName:ef8961d7-c2fb-4066-ad2b-feb9b153a2b8 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:30.940811136 +0000 UTC m=+37.068897804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls") pod "image-registry-77f64d5f4b-qxgph" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8") : secret "image-registry-tls" not found
Jul 07 02:18:30.395549 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.395473    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:30.396064 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.395472    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:30.398741 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.398716    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 02:18:30.398861 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.398764    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 02:18:30.399021 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.399002    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 02:18:30.399988 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.399970    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sv29t\""
Jul 07 02:18:30.695041 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.695013    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"]
Jul 07 02:18:30.697807 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.697777    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kh65r" event={"ID":"5244c428-40dd-4db7-acb9-5723befa306b","Type":"ContainerStarted","Data":"ff47ec224484f325bfeee6c389873e278ffa48f9f6d03442e65d47e0c5ad4d48"}
Jul 07 02:18:30.697956 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.697899    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:30.701121 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.701096    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 02:18:30.701121 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.701099    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dm26g\""
Jul 07 02:18:30.702460 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.702415    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 02:18:30.710078 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.710056    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"]
Jul 07 02:18:30.847564 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.847513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f0eb21a1-d159-4e50-8f8a-548cca607197-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:30.847728 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.847651    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:30.949017 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.948924    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:30.949017 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.948970    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f0eb21a1-d159-4e50-8f8a-548cca607197-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:30.949235 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:30.949085    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:18:30.949235 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.949136    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:30.949235 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:30.949161    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls podName:ca58ca66-f969-4f25-85ff-b95d5ac9595b nodeName:}" failed. No retries permitted until 2026-07-07 02:18:32.949146164 +0000 UTC m=+39.077232833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls") pod "dns-default-xqnxc" (UID: "ca58ca66-f969-4f25-85ff-b95d5ac9595b") : secret "dns-default-metrics-tls" not found
Jul 07 02:18:30.949235 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.949220    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:30.949235 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:30.949231    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:18:30.949564 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:30.949246    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f64d5f4b-qxgph: secret "image-registry-tls" not found
Jul 07 02:18:30.949564 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:30.949304    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls podName:ef8961d7-c2fb-4066-ad2b-feb9b153a2b8 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:32.949287321 +0000 UTC m=+39.077374007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls") pod "image-registry-77f64d5f4b-qxgph" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8") : secret "image-registry-tls" not found
Jul 07 02:18:30.949564 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:30.949363    2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 02:18:30.949564 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:30.949420    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert podName:f0eb21a1-d159-4e50-8f8a-548cca607197 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:31.449402269 +0000 UTC m=+37.577488945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tr6tr" (UID: "f0eb21a1-d159-4e50-8f8a-548cca607197") : secret "networking-console-plugin-cert" not found
Jul 07 02:18:30.949705 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:30.949599    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f0eb21a1-d159-4e50-8f8a-548cca607197-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:31.097456 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.097402    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"]
Jul 07 02:18:31.099890 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.099861    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"
Jul 07 02:18:31.106950 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.106489    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wdnx5\""
Jul 07 02:18:31.106950 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.106848    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 02:18:31.107406 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.107387    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 02:18:31.121494 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.121460    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"]
Jul 07 02:18:31.232336 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.232250    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ffpwx"]
Jul 07 02:18:31.234426 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.234403    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.238440 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.238417    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 02:18:31.238594 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.238417    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 02:18:31.239088 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.239070    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 02:18:31.239545 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.239529    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vjbfk\""
Jul 07 02:18:31.239828 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.239813    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 02:18:31.251316 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.251289    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz8xq\" (UniqueName: \"kubernetes.io/projected/4cdfd384-4f36-4a27-8a92-a351a530f803-kube-api-access-lz8xq\") pod \"migrator-d57d58cc9-sm9dm\" (UID: \"4cdfd384-4f36-4a27-8a92-a351a530f803\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"
Jul 07 02:18:31.259076 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.259046    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ffpwx"]
Jul 07 02:18:31.352041 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.352008    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.352041 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.352045    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.352303 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.352074    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz8xq\" (UniqueName: \"kubernetes.io/projected/4cdfd384-4f36-4a27-8a92-a351a530f803-kube-api-access-lz8xq\") pod \"migrator-d57d58cc9-sm9dm\" (UID: \"4cdfd384-4f36-4a27-8a92-a351a530f803\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"
Jul 07 02:18:31.352364 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.352337    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-crio-socket\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.352433 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.352379    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-data-volume\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.352506 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.352480    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hkct\" (UniqueName: \"kubernetes.io/projected/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-api-access-4hkct\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.365515 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.365484    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz8xq\" (UniqueName: \"kubernetes.io/projected/4cdfd384-4f36-4a27-8a92-a351a530f803-kube-api-access-lz8xq\") pod \"migrator-d57d58cc9-sm9dm\" (UID: \"4cdfd384-4f36-4a27-8a92-a351a530f803\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"
Jul 07 02:18:31.409385 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.409342    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"
Jul 07 02:18:31.453853 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.453813    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-crio-socket\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.453853 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.453854    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-data-volume\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.454072 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.453880    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hkct\" (UniqueName: \"kubernetes.io/projected/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-api-access-4hkct\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.454072 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.453912    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:31.454072 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.453951    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.454072 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.453978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.454090    2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.454108    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-crio-socket\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.454116    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.454151    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.454157    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert podName:f0eb21a1-d159-4e50-8f8a-548cca607197 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:32.454137209 +0000 UTC m=+38.582223896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tr6tr" (UID: "f0eb21a1-d159-4e50-8f8a-548cca607197") : secret "networking-console-plugin-cert" not found
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.454196    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:31.954176291 +0000 UTC m=+38.082262978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.454216    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:31.95420638 +0000 UTC m=+38.082293054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : configmap "kube-rbac-proxy" not found
Jul 07 02:18:31.454250 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.454234    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-data-volume\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.467580 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.467547    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hkct\" (UniqueName: \"kubernetes.io/projected/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-api-access-4hkct\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.903986 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.903951    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm"]
Jul 07 02:18:31.907773 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:31.907743    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdfd384_4f36_4a27_8a92_a351a530f803.slice/crio-ab14179b2384b52533d690ee91238fa01879a6c87dca5f49e5e46cf25525de53 WatchSource:0}: Error finding container ab14179b2384b52533d690ee91238fa01879a6c87dca5f49e5e46cf25525de53: Status 404 returned error can't find the container with id ab14179b2384b52533d690ee91238fa01879a6c87dca5f49e5e46cf25525de53
Jul 07 02:18:31.957887 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.957852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.957887 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:31.957891    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:31.958084 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.957964    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:18:31.958084 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.958030    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:32.958013389 +0000 UTC m=+39.086100057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : configmap "kube-rbac-proxy" not found
Jul 07 02:18:31.958084 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.958070    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:31.958189 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:31.958129    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:32.958116202 +0000 UTC m=+39.086202871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:32.462675 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.462639    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:32.463153 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.462823    2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 02:18:32.463153 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.462908    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert podName:f0eb21a1-d159-4e50-8f8a-548cca607197 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:34.462888172 +0000 UTC m=+40.590974845 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tr6tr" (UID: "f0eb21a1-d159-4e50-8f8a-548cca607197") : secret "networking-console-plugin-cert" not found
Jul 07 02:18:32.705679 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.705641    2585 generic.go:356] "Generic (PLEG): container finished" podID="e0050f4f-59a5-4548-a62d-cb7c61cff8d1" containerID="c8975e2d25fdc985098af39dfaf434238dc20177a39e0d4849df64ac8930c65a" exitCode=0
Jul 07 02:18:32.705840 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.705683    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerDied","Data":"c8975e2d25fdc985098af39dfaf434238dc20177a39e0d4849df64ac8930c65a"}
Jul 07 02:18:32.707122 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.707097    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm" event={"ID":"4cdfd384-4f36-4a27-8a92-a351a530f803","Type":"ContainerStarted","Data":"ab14179b2384b52533d690ee91238fa01879a6c87dca5f49e5e46cf25525de53"}
Jul 07 02:18:32.968131 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.968036    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:32.968131 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.968088    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:32.968358 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.968150    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:32.968358 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:32.968185    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:32.968358 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968342    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:18:32.968358 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968358    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f64d5f4b-qxgph: secret "image-registry-tls" not found
Jul 07 02:18:32.968557 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968405    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls podName:ef8961d7-c2fb-4066-ad2b-feb9b153a2b8 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:36.968389664 +0000 UTC m=+43.096476336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls") pod "image-registry-77f64d5f4b-qxgph" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8") : secret "image-registry-tls" not found
Jul 07 02:18:32.968909 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968737    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:32.968909 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968790    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:18:32.968909 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968804    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:34.968785108 +0000 UTC m=+41.096871784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:32.968909 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968840    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls podName:ca58ca66-f969-4f25-85ff-b95d5ac9595b nodeName:}" failed. No retries permitted until 2026-07-07 02:18:36.968824918 +0000 UTC m=+43.096911589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls") pod "dns-default-xqnxc" (UID: "ca58ca66-f969-4f25-85ff-b95d5ac9595b") : secret "dns-default-metrics-tls" not found
Jul 07 02:18:32.968909 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968853    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 02:18:32.968909 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:32.968887    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:34.968875915 +0000 UTC m=+41.096962587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : configmap "kube-rbac-proxy" not found
Jul 07 02:18:33.419606 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.419512    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qbffq_255f2d58-2da4-4f7c-8e74-e2b5d583f2a2/dns-node-resolver/0.log"
Jul 07 02:18:33.685114 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.685079    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-2fd9v"]
Jul 07 02:18:33.703752 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.703712    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-2fd9v"]
Jul 07 02:18:33.703894 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.703845    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.706896 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.706871    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 02:18:33.708640 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.708613    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 02:18:33.708778 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.708671    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 02:18:33.708778 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.708724    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 02:18:33.708943 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.708924    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vbl98\""
Jul 07 02:18:33.714772 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.714545    2585 generic.go:356] "Generic (PLEG): container finished" podID="e0050f4f-59a5-4548-a62d-cb7c61cff8d1" containerID="81e8140c9f94784f78be2e19e0cbd116178ef723fa2a1377562da5b465b6ba4a" exitCode=0
Jul 07 02:18:33.714772 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.714616    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerDied","Data":"81e8140c9f94784f78be2e19e0cbd116178ef723fa2a1377562da5b465b6ba4a"}
Jul 07 02:18:33.878320 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.878284    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-signing-cabundle\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.878526 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.878388    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctz8p\" (UniqueName: \"kubernetes.io/projected/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-kube-api-access-ctz8p\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.878575 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.878524    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-signing-key\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.980077 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.979995    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-signing-cabundle\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.980077 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.980056    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctz8p\" (UniqueName: \"kubernetes.io/projected/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-kube-api-access-ctz8p\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.980265 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.980162    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-signing-key\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.980876 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.980849    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-signing-cabundle\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.983788 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.983625    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-signing-key\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:33.988677 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:33.988649    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctz8p\" (UniqueName: \"kubernetes.io/projected/a97f0832-1377-4eb0-8bd9-15cd3f3dd14a-kube-api-access-ctz8p\") pod \"service-ca-758f599c5-2fd9v\" (UID: \"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a\") " pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:34.015871 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:34.015833    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-2fd9v"
Jul 07 02:18:34.484616 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:34.484575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:34.484786 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:34.484696    2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 02:18:34.484786 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:34.484749    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert podName:f0eb21a1-d159-4e50-8f8a-548cca607197 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:38.484734805 +0000 UTC m=+44.612821478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tr6tr" (UID: "f0eb21a1-d159-4e50-8f8a-548cca607197") : secret "networking-console-plugin-cert" not found
Jul 07 02:18:34.623561 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:34.623531    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gkgb6_278a6bcd-e480-4cab-9a83-a53245336588/node-ca/0.log"
Jul 07 02:18:34.782099 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:34.781956    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-2fd9v"]
Jul 07 02:18:34.784695 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:34.784670    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97f0832_1377_4eb0_8bd9_15cd3f3dd14a.slice/crio-10a6ec69abd1bac61b895c377543653fae2efc1c2db9ae437d3895095d760159 WatchSource:0}: Error finding container 10a6ec69abd1bac61b895c377543653fae2efc1c2db9ae437d3895095d760159: Status 404 returned error can't find the container with id 10a6ec69abd1bac61b895c377543653fae2efc1c2db9ae437d3895095d760159
Jul 07 02:18:34.989228 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:34.989194    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:34.989375 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:34.989240    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:34.989470 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:34.989429    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:34.989535 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:34.989524    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:38.989503683 +0000 UTC m=+45.117590355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:34.989782 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:34.989762    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:35.724420 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.724383    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" event={"ID":"e0050f4f-59a5-4548-a62d-cb7c61cff8d1","Type":"ContainerStarted","Data":"483318572f5de31a550443f8b896b9aa9b2bbd31dfb03c644a0da1cb59b4d67d"}
Jul 07 02:18:35.726330 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.726303    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kh65r" event={"ID":"5244c428-40dd-4db7-acb9-5723befa306b","Type":"ContainerStarted","Data":"e6e5513f0fff19c0a6dd3a76a59131863962c1c7640fbf6c7ba7eb9e2066ca62"}
Jul 07 02:18:35.728282 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.728097    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm" event={"ID":"4cdfd384-4f36-4a27-8a92-a351a530f803","Type":"ContainerStarted","Data":"498a2e21caeaaf42281c6be35ef1e1e22791798de37ea5064a5c742d1b1b35d7"}
Jul 07 02:18:35.728282 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.728152    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm" event={"ID":"4cdfd384-4f36-4a27-8a92-a351a530f803","Type":"ContainerStarted","Data":"d6a75e967eaf2c4787888b673674eade4724c7e1889149691c3d33ffe7bbd0d2"}
Jul 07 02:18:35.729147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.729121    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-2fd9v" event={"ID":"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a","Type":"ContainerStarted","Data":"10a6ec69abd1bac61b895c377543653fae2efc1c2db9ae437d3895095d760159"}
Jul 07 02:18:35.750896 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.750836    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s6z8n" podStartSLOduration=10.605881348 podStartE2EDuration="41.750815282s" totalImagesPullingTime="31.144933934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:35.749186347 +0000 UTC m=+41.877273030" watchObservedRunningTime="2026-07-07 02:18:35.750815282 +0000 UTC m=+41.878901973"
Jul 07 02:18:35.767767 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.767709    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kh65r" podStartSLOduration=1.8649363220000001 podStartE2EDuration="6.767686528s" totalImagesPullingTime="4.902750206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:35.767145208 +0000 UTC m=+41.895231911" watchObservedRunningTime="2026-07-07 02:18:35.767686528 +0000 UTC m=+41.895773222"
Jul 07 02:18:35.790164 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.790104    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sm9dm" podStartSLOduration=2.0658768849999998 podStartE2EDuration="4.790087333s" totalImagesPullingTime="2.724210448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:35.789840886 +0000 UTC m=+41.917927577" watchObservedRunningTime="2026-07-07 02:18:35.790087333 +0000 UTC m=+41.918174023"
Jul 07 02:18:35.824155 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:35.824125    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sm9dm_4cdfd384-4f36-4a27-8a92-a351a530f803/graceful-termination/0.log"
Jul 07 02:18:37.010670 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:37.010625    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:37.011094 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:37.010694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:37.011094 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:37.010803    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 02:18:37.011094 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:37.010811    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 02:18:37.011094 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:37.010838    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f64d5f4b-qxgph: secret "image-registry-tls" not found
Jul 07 02:18:37.011094 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:37.010892    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls podName:ca58ca66-f969-4f25-85ff-b95d5ac9595b nodeName:}" failed. No retries permitted until 2026-07-07 02:18:45.010872339 +0000 UTC m=+51.138959012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls") pod "dns-default-xqnxc" (UID: "ca58ca66-f969-4f25-85ff-b95d5ac9595b") : secret "dns-default-metrics-tls" not found
Jul 07 02:18:37.011094 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:37.010908    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls podName:ef8961d7-c2fb-4066-ad2b-feb9b153a2b8 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:45.010902141 +0000 UTC m=+51.138988810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls") pod "image-registry-77f64d5f4b-qxgph" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8") : secret "image-registry-tls" not found
Jul 07 02:18:37.735821 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:37.735792    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-2fd9v" event={"ID":"a97f0832-1377-4eb0-8bd9-15cd3f3dd14a","Type":"ContainerStarted","Data":"f1aa0bbd8feea570aafe1fe887c8b413360fa0a9b9882d460ffcedb5125a0fa0"}
Jul 07 02:18:38.524673 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:38.524629    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:38.525055 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:38.524799    2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 02:18:38.525055 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:38.524875    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert podName:f0eb21a1-d159-4e50-8f8a-548cca607197 nodeName:}" failed. No retries permitted until 2026-07-07 02:18:46.524855922 +0000 UTC m=+52.652942591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tr6tr" (UID: "f0eb21a1-d159-4e50-8f8a-548cca607197") : secret "networking-console-plugin-cert" not found
Jul 07 02:18:38.758011 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:38.757947    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-2fd9v" podStartSLOduration=2.9294103160000002 podStartE2EDuration="5.757925373s" totalImagesPullingTime="2.828515057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:38.756959382 +0000 UTC m=+44.885046073" watchObservedRunningTime="2026-07-07 02:18:38.757925373 +0000 UTC m=+44.886012069"
Jul 07 02:18:39.030206 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:39.030164    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:39.030639 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:39.030348    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:39.030791 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:39.030780    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls podName:697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d nodeName:}" failed. No retries permitted until 2026-07-07 02:18:47.030757976 +0000 UTC m=+53.158844649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ffpwx" (UID: "697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d") : secret "insights-runtime-extractor-tls" not found
Jul 07 02:18:45.083523 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.083482    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:45.083979 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.083534    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:45.085959 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.085932    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca58ca66-f969-4f25-85ff-b95d5ac9595b-metrics-tls\") pod \"dns-default-xqnxc\" (UID: \"ca58ca66-f969-4f25-85ff-b95d5ac9595b\") " pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:45.086201 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.086178    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"image-registry-77f64d5f4b-qxgph\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") " pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:45.143515 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.143473    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:45.167769 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.167747    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:45.310013 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.309983    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77f64d5f4b-qxgph"]
Jul 07 02:18:45.313330 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:45.313292    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8961d7_c2fb_4066_ad2b_feb9b153a2b8.slice/crio-6a3982abb8a8359fc6f198cf724e7fd96d72f2f81215833eaebb9088f542203e WatchSource:0}: Error finding container 6a3982abb8a8359fc6f198cf724e7fd96d72f2f81215833eaebb9088f542203e: Status 404 returned error can't find the container with id 6a3982abb8a8359fc6f198cf724e7fd96d72f2f81215833eaebb9088f542203e
Jul 07 02:18:45.330878 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.330847    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xqnxc"]
Jul 07 02:18:45.333592 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:45.333529    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca58ca66_f969_4f25_85ff_b95d5ac9595b.slice/crio-7f61f820db270976336bbfbf3d9d46c22c0f065735288735a67deef1b1716efd WatchSource:0}: Error finding container 7f61f820db270976336bbfbf3d9d46c22c0f065735288735a67deef1b1716efd: Status 404 returned error can't find the container with id 7f61f820db270976336bbfbf3d9d46c22c0f065735288735a67deef1b1716efd
Jul 07 02:18:45.759524 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.759483    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xqnxc" event={"ID":"ca58ca66-f969-4f25-85ff-b95d5ac9595b","Type":"ContainerStarted","Data":"7f61f820db270976336bbfbf3d9d46c22c0f065735288735a67deef1b1716efd"}
Jul 07 02:18:45.761037 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.761010    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph" event={"ID":"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8","Type":"ContainerStarted","Data":"a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202"}
Jul 07 02:18:45.761176 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.761047    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph" event={"ID":"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8","Type":"ContainerStarted","Data":"6a3982abb8a8359fc6f198cf724e7fd96d72f2f81215833eaebb9088f542203e"}
Jul 07 02:18:45.761176 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.761144    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:18:45.780697 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:45.780645    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph" podStartSLOduration=36.780629896 podStartE2EDuration="36.780629896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:45.780146078 +0000 UTC m=+51.908232768" watchObservedRunningTime="2026-07-07 02:18:45.780629896 +0000 UTC m=+51.908716586"
Jul 07 02:18:46.596851 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:46.596807    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:18:46.597301 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:46.596978    2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 02:18:46.597301 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:18:46.597051    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert podName:f0eb21a1-d159-4e50-8f8a-548cca607197 nodeName:}" failed. No retries permitted until 2026-07-07 02:19:02.597034586 +0000 UTC m=+68.725121255 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tr6tr" (UID: "f0eb21a1-d159-4e50-8f8a-548cca607197") : secret "networking-console-plugin-cert" not found
Jul 07 02:18:47.102047 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:47.102008    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:47.105066 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:47.105034    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ffpwx\" (UID: \"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d\") " pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:47.144999 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:47.144959    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ffpwx"
Jul 07 02:18:47.676922 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:47.676900    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ffpwx"]
Jul 07 02:18:47.679640 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:47.679599    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697bf4aa_e2cd_4fc2_a2e2_a7ca6d9e300d.slice/crio-88bb47c998ed2037280e8c5f17d801265548d639a2269cb0a3e9003ae58f3e52 WatchSource:0}: Error finding container 88bb47c998ed2037280e8c5f17d801265548d639a2269cb0a3e9003ae58f3e52: Status 404 returned error can't find the container with id 88bb47c998ed2037280e8c5f17d801265548d639a2269cb0a3e9003ae58f3e52
Jul 07 02:18:47.771523 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:47.771485    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ffpwx" event={"ID":"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d","Type":"ContainerStarted","Data":"88bb47c998ed2037280e8c5f17d801265548d639a2269cb0a3e9003ae58f3e52"}
Jul 07 02:18:48.779324 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:48.779217    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xqnxc" event={"ID":"ca58ca66-f969-4f25-85ff-b95d5ac9595b","Type":"ContainerStarted","Data":"36b9ed0ecd4919c22b9778d7ce37ae712d9557ddfbfec4ff8bdfd1c9ad1bca69"}
Jul 07 02:18:48.779324 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:48.779284    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xqnxc" event={"ID":"ca58ca66-f969-4f25-85ff-b95d5ac9595b","Type":"ContainerStarted","Data":"c86fd0d30a791690cfff8c5003232e0928cc5c6191d315c374eb58465d483b44"}
Jul 07 02:18:48.779872 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:48.779406    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:48.798079 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:48.798014    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xqnxc" podStartSLOduration=17.249164112 podStartE2EDuration="19.797992129s" totalImagesPullingTime="2.548828017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:48.79628959 +0000 UTC m=+54.924376281" watchObservedRunningTime="2026-07-07 02:18:48.797992129 +0000 UTC m=+54.926078822"
Jul 07 02:18:51.791561 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:51.791523    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ffpwx" event={"ID":"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d","Type":"ContainerStarted","Data":"d69d211e20986960e97695dc4b68ac02256e8ee0881a0e5631216863a7612165"}
Jul 07 02:18:52.796886 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:52.796847    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ffpwx" event={"ID":"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d","Type":"ContainerStarted","Data":"a201407d6257e11002d30551fe0855250581a03eda9f6b310f2605639b664aea"}
Jul 07 02:18:55.808490 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:55.808431    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ffpwx" event={"ID":"697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d","Type":"ContainerStarted","Data":"cabcc687f231f65ec6eb061f4ad7e0cb870fc9921f9c105bdebb11d397a98017"}
Jul 07 02:18:55.826648 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:55.826599    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ffpwx" podStartSLOduration=17.579162038 podStartE2EDuration="24.826584069s" totalImagesPullingTime="7.247422031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:18:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:55.82596775 +0000 UTC m=+61.954054440" watchObservedRunningTime="2026-07-07 02:18:55.826584069 +0000 UTC m=+61.954670772"
Jul 07 02:18:56.694786 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:56.694761    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s4lcd"
Jul 07 02:18:57.817291 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.817256    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77f64d5f4b-qxgph"]
Jul 07 02:18:57.821505 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.820410    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-44thl"]
Jul 07 02:18:57.823398 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.823377    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-44thl"
Jul 07 02:18:57.828789 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.828770    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 02:18:57.828976 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.828950    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 02:18:57.829064 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.828954    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-r65s7\""
Jul 07 02:18:57.850590 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.850562    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-44thl"]
Jul 07 02:18:57.891638 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.891610    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdvlm\" (UniqueName: \"kubernetes.io/projected/0650f4f3-6638-486c-b6a8-92eff0dd11a1-kube-api-access-sdvlm\") pod \"downloads-685bfb5f64-44thl\" (UID: \"0650f4f3-6638-486c-b6a8-92eff0dd11a1\") " pod="openshift-console/downloads-685bfb5f64-44thl"
Jul 07 02:18:57.920824 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.920786    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c849fb7f4-f7sxk"]
Jul 07 02:18:57.923027 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.923005    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.943007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.942977    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c849fb7f4-f7sxk"]
Jul 07 02:18:57.992340 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992305    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgbjn\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-kube-api-access-fgbjn\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992340 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992347    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3e54dda5-0900-4d35-bda9-b49eef671a8c-installation-pull-secrets\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992568 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992373    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-bound-sa-token\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992568 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992430    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-registry-tls\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992568 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992517    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3e54dda5-0900-4d35-bda9-b49eef671a8c-ca-trust-extracted\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992568 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992554    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3e54dda5-0900-4d35-bda9-b49eef671a8c-image-registry-private-configuration\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992692 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992574    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3e54dda5-0900-4d35-bda9-b49eef671a8c-registry-certificates\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992692 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992600    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e54dda5-0900-4d35-bda9-b49eef671a8c-trusted-ca\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:57.992692 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:57.992626    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdvlm\" (UniqueName: \"kubernetes.io/projected/0650f4f3-6638-486c-b6a8-92eff0dd11a1-kube-api-access-sdvlm\") pod \"downloads-685bfb5f64-44thl\" (UID: \"0650f4f3-6638-486c-b6a8-92eff0dd11a1\") " pod="openshift-console/downloads-685bfb5f64-44thl"
Jul 07 02:18:58.003289 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.003259    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdvlm\" (UniqueName: \"kubernetes.io/projected/0650f4f3-6638-486c-b6a8-92eff0dd11a1-kube-api-access-sdvlm\") pod \"downloads-685bfb5f64-44thl\" (UID: \"0650f4f3-6638-486c-b6a8-92eff0dd11a1\") " pod="openshift-console/downloads-685bfb5f64-44thl"
Jul 07 02:18:58.093358 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093276    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgbjn\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-kube-api-access-fgbjn\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.093358 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093314    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3e54dda5-0900-4d35-bda9-b49eef671a8c-installation-pull-secrets\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.093358 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093332    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-bound-sa-token\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.093652 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093361    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-registry-tls\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.093652 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093406    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3e54dda5-0900-4d35-bda9-b49eef671a8c-ca-trust-extracted\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.093652 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093464    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3e54dda5-0900-4d35-bda9-b49eef671a8c-image-registry-private-configuration\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.093652 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093489    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3e54dda5-0900-4d35-bda9-b49eef671a8c-registry-certificates\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.093652 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093507    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e54dda5-0900-4d35-bda9-b49eef671a8c-trusted-ca\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.094014 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.093963    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3e54dda5-0900-4d35-bda9-b49eef671a8c-ca-trust-extracted\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.094496 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.094474    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3e54dda5-0900-4d35-bda9-b49eef671a8c-registry-certificates\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.095075 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.095048    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e54dda5-0900-4d35-bda9-b49eef671a8c-trusted-ca\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.096034 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.096016    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3e54dda5-0900-4d35-bda9-b49eef671a8c-installation-pull-secrets\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.096127 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.096078    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3e54dda5-0900-4d35-bda9-b49eef671a8c-image-registry-private-configuration\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.096127 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.096117    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-registry-tls\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.105508 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.105482    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-bound-sa-token\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.106409 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.106379    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgbjn\" (UniqueName: \"kubernetes.io/projected/3e54dda5-0900-4d35-bda9-b49eef671a8c-kube-api-access-fgbjn\") pod \"image-registry-6c849fb7f4-f7sxk\" (UID: \"3e54dda5-0900-4d35-bda9-b49eef671a8c\") " pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.134246 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.134209    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-44thl"
Jul 07 02:18:58.231881 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.231854    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.268130 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.268099    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-44thl"]
Jul 07 02:18:58.272662 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:58.272632    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0650f4f3_6638_486c_b6a8_92eff0dd11a1.slice/crio-649ebea99d53eb8737551066fe8c6ca97aefc14765ae752e75d05e0ad1f06e0d WatchSource:0}: Error finding container 649ebea99d53eb8737551066fe8c6ca97aefc14765ae752e75d05e0ad1f06e0d: Status 404 returned error can't find the container with id 649ebea99d53eb8737551066fe8c6ca97aefc14765ae752e75d05e0ad1f06e0d
Jul 07 02:18:58.367187 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.367114    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c849fb7f4-f7sxk"]
Jul 07 02:18:58.370040 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:58.370007    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e54dda5_0900_4d35_bda9_b49eef671a8c.slice/crio-ebc49bdf5e6d9a640f1059b7c3c99a7fd1132d9f83c261fc50404e1894f2e96c WatchSource:0}: Error finding container ebc49bdf5e6d9a640f1059b7c3c99a7fd1132d9f83c261fc50404e1894f2e96c: Status 404 returned error can't find the container with id ebc49bdf5e6d9a640f1059b7c3c99a7fd1132d9f83c261fc50404e1894f2e96c
Jul 07 02:18:58.785039 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.785011    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xqnxc"
Jul 07 02:18:58.819661 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.819619    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk" event={"ID":"3e54dda5-0900-4d35-bda9-b49eef671a8c","Type":"ContainerStarted","Data":"8f2bf3e54babc57b853073dfec74270d099d8724d240e2d86816e9d5f2f2a064"}
Jul 07 02:18:58.820126 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.819682    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk" event={"ID":"3e54dda5-0900-4d35-bda9-b49eef671a8c","Type":"ContainerStarted","Data":"ebc49bdf5e6d9a640f1059b7c3c99a7fd1132d9f83c261fc50404e1894f2e96c"}
Jul 07 02:18:58.820126 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.819714    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:18:58.821005 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.820980    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-44thl" event={"ID":"0650f4f3-6638-486c-b6a8-92eff0dd11a1","Type":"ContainerStarted","Data":"649ebea99d53eb8737551066fe8c6ca97aefc14765ae752e75d05e0ad1f06e0d"}
Jul 07 02:18:58.930883 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:58.930831    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk" podStartSLOduration=1.930813181 podStartE2EDuration="1.930813181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:58.927135603 +0000 UTC m=+65.055222296" watchObservedRunningTime="2026-07-07 02:18:58.930813181 +0000 UTC m=+65.058899873"
Jul 07 02:18:59.202344 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.202304    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:59.202580 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.202384    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:59.205656 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.205632    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 02:18:59.205763 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.205635    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 02:18:59.215483 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.215437    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c980fc15-667f-435a-8215-11d614462009-metrics-certs\") pod \"network-metrics-daemon-lklr2\" (UID: \"c980fc15-667f-435a-8215-11d614462009\") " pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:59.215610 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.215486    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25e0d312-fbb7-40bd-93b3-ce1dd410426f-original-pull-secret\") pod \"global-pull-secret-syncer-n82t2\" (UID: \"25e0d312-fbb7-40bd-93b3-ce1dd410426f\") " pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:59.302946 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.302907    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:59.306034 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.306011    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 02:18:59.316713 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.316689    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 02:18:59.326875 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.326845    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9szf\" (UniqueName: \"kubernetes.io/projected/8adfce1b-afab-453b-a0ad-97c3e0dbdf6a-kube-api-access-l9szf\") pod \"network-check-target-bfq5k\" (UID: \"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a\") " pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:59.407307 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.407275    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kthkt\""
Jul 07 02:18:59.414001 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.413976    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lklr2"
Jul 07 02:18:59.510148 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.510121    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sv29t\""
Jul 07 02:18:59.512767 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.512744    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n82t2"
Jul 07 02:18:59.517857 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.517667    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:18:59.557434 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.557412    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lklr2"]
Jul 07 02:18:59.561403 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:59.560310    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc980fc15_667f_435a_8215_11d614462009.slice/crio-2a7553f9a2ca82276ef7a9c79cfe9dd414fc660dfa6c25aa3aa7b3a52aa7687c WatchSource:0}: Error finding container 2a7553f9a2ca82276ef7a9c79cfe9dd414fc660dfa6c25aa3aa7b3a52aa7687c: Status 404 returned error can't find the container with id 2a7553f9a2ca82276ef7a9c79cfe9dd414fc660dfa6c25aa3aa7b3a52aa7687c
Jul 07 02:18:59.669896 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.669867    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bfq5k"]
Jul 07 02:18:59.672237 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:59.672213    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8adfce1b_afab_453b_a0ad_97c3e0dbdf6a.slice/crio-b5592d038da3c23cd39ffa741af89cf3edf97d4f377171dedf6150ea328a473f WatchSource:0}: Error finding container b5592d038da3c23cd39ffa741af89cf3edf97d4f377171dedf6150ea328a473f: Status 404 returned error can't find the container with id b5592d038da3c23cd39ffa741af89cf3edf97d4f377171dedf6150ea328a473f
Jul 07 02:18:59.682037 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.682018    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n82t2"]
Jul 07 02:18:59.684185 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:18:59.684164    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e0d312_fbb7_40bd_93b3_ce1dd410426f.slice/crio-e45dbb1b7330d37d1c467fcc41e8de6b0ac144ab07607092036276a80f3de31b WatchSource:0}: Error finding container e45dbb1b7330d37d1c467fcc41e8de6b0ac144ab07607092036276a80f3de31b: Status 404 returned error can't find the container with id e45dbb1b7330d37d1c467fcc41e8de6b0ac144ab07607092036276a80f3de31b
Jul 07 02:18:59.825576 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.825488    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bfq5k" event={"ID":"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a","Type":"ContainerStarted","Data":"b5592d038da3c23cd39ffa741af89cf3edf97d4f377171dedf6150ea328a473f"}
Jul 07 02:18:59.826622 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.826599    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n82t2" event={"ID":"25e0d312-fbb7-40bd-93b3-ce1dd410426f","Type":"ContainerStarted","Data":"e45dbb1b7330d37d1c467fcc41e8de6b0ac144ab07607092036276a80f3de31b"}
Jul 07 02:18:59.827614 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:18:59.827595    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lklr2" event={"ID":"c980fc15-667f-435a-8215-11d614462009","Type":"ContainerStarted","Data":"2a7553f9a2ca82276ef7a9c79cfe9dd414fc660dfa6c25aa3aa7b3a52aa7687c"}
Jul 07 02:19:01.710580 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.710545    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66c5f95f59-q8p2f"]
Jul 07 02:19:01.729218 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.729174    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c5f95f59-q8p2f"]
Jul 07 02:19:01.729392 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.729293    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.733074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.732870    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 02:19:01.733074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.732870    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mmk7l\""
Jul 07 02:19:01.733301 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.733045    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 02:19:01.734388 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.734364    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 02:19:01.734601 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.734582    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 02:19:01.734693 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.734588    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 02:19:01.828142 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.828065    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkrjn\" (UniqueName: \"kubernetes.io/projected/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-kube-api-access-nkrjn\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.828142 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.828144    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-service-ca\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.828394 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.828173    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-config\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.828394 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.828209    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-oauth-serving-cert\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.828394 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.828260    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-oauth-config\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.828394 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.828326    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-serving-cert\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.838330 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.838293    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lklr2" event={"ID":"c980fc15-667f-435a-8215-11d614462009","Type":"ContainerStarted","Data":"a901dd3dffde072f9c75cb294d1bec7e5b3ba069a73b1dc421944788bbb2a14c"}
Jul 07 02:19:01.838544 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.838361    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lklr2" event={"ID":"c980fc15-667f-435a-8215-11d614462009","Type":"ContainerStarted","Data":"5f9077af8b6dceb22376d31bf5634b6981b08f1c384c50564fdf9178c12ac9d6"}
Jul 07 02:19:01.857746 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.857679    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lklr2" podStartSLOduration=66.597304943 podStartE2EDuration="1m7.857662677s" totalImagesPullingTime="1.260357734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:01.85673865 +0000 UTC m=+67.984825354" watchObservedRunningTime="2026-07-07 02:19:01.857662677 +0000 UTC m=+67.985749367"
Jul 07 02:19:01.929556 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.929520    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkrjn\" (UniqueName: \"kubernetes.io/projected/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-kube-api-access-nkrjn\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.929704 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.929577    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-service-ca\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.929704 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.929595    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-config\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.929704 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.929619    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-oauth-serving-cert\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.929704 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.929643    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-oauth-config\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.929704 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.929675    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-serving-cert\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.931090 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.931062    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-config\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.931224 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.931159    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-service-ca\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.931224 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.931170    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-oauth-serving-cert\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.933709 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.933672    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-oauth-config\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.933709 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.933692    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-serving-cert\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:01.955578 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:01.955514    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkrjn\" (UniqueName: \"kubernetes.io/projected/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-kube-api-access-nkrjn\") pod \"console-66c5f95f59-q8p2f\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") " pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:02.042833 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:02.042533    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:02.634337 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:02.634301    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:19:02.645139 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:02.645073    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f0eb21a1-d159-4e50-8f8a-548cca607197-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tr6tr\" (UID: \"f0eb21a1-d159-4e50-8f8a-548cca607197\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:19:02.811966 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:02.811734    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dm26g\""
Jul 07 02:19:02.819517 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:02.819235    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"
Jul 07 02:19:04.924015 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:04.923968    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c5f95f59-q8p2f"]
Jul 07 02:19:04.927099 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:19:04.927053    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dad02ae_1e12_4b6a_b0c7_5c74d553e612.slice/crio-468a5179ad3cd306ecf4ef9e8d0a83dc9acb6475a67ea68b2f444ffeebc55ec9 WatchSource:0}: Error finding container 468a5179ad3cd306ecf4ef9e8d0a83dc9acb6475a67ea68b2f444ffeebc55ec9: Status 404 returned error can't find the container with id 468a5179ad3cd306ecf4ef9e8d0a83dc9acb6475a67ea68b2f444ffeebc55ec9
Jul 07 02:19:04.946258 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:04.946227    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr"]
Jul 07 02:19:04.950803 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:19:04.950770    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0eb21a1_d159_4e50_8f8a_548cca607197.slice/crio-98952e64c43159a7c7a2d2bc1465f2aec416732ee647b7efeb5f04645c727653 WatchSource:0}: Error finding container 98952e64c43159a7c7a2d2bc1465f2aec416732ee647b7efeb5f04645c727653: Status 404 returned error can't find the container with id 98952e64c43159a7c7a2d2bc1465f2aec416732ee647b7efeb5f04645c727653
Jul 07 02:19:05.866524 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:05.865704    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bfq5k" event={"ID":"8adfce1b-afab-453b-a0ad-97c3e0dbdf6a","Type":"ContainerStarted","Data":"0d1ce56af2779d6a72eda0ed8c6308fd6e0ce37d2a608a590ed9d54069382bc4"}
Jul 07 02:19:05.866524 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:05.866439    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:19:05.870286 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:05.870234    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n82t2" event={"ID":"25e0d312-fbb7-40bd-93b3-ce1dd410426f","Type":"ContainerStarted","Data":"aa08d7e14d0d5750d44b0e236d1007e888f4c1db0d820af036805852f21692ad"}
Jul 07 02:19:05.872075 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:05.871879    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr" event={"ID":"f0eb21a1-d159-4e50-8f8a-548cca607197","Type":"ContainerStarted","Data":"98952e64c43159a7c7a2d2bc1465f2aec416732ee647b7efeb5f04645c727653"}
Jul 07 02:19:05.873462 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:05.873416    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c5f95f59-q8p2f" event={"ID":"9dad02ae-1e12-4b6a-b0c7-5c74d553e612","Type":"ContainerStarted","Data":"468a5179ad3cd306ecf4ef9e8d0a83dc9acb6475a67ea68b2f444ffeebc55ec9"}
Jul 07 02:19:05.890712 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:05.890053    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bfq5k" podStartSLOduration=66.801018094 podStartE2EDuration="1m11.890032009s" totalImagesPullingTime="5.089013915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:05.889085499 +0000 UTC m=+72.017172223" watchObservedRunningTime="2026-07-07 02:19:05.890032009 +0000 UTC m=+72.018118700"
Jul 07 02:19:05.922949 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:05.922062    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n82t2" podStartSLOduration=66.834480699 podStartE2EDuration="1m11.922039875s" totalImagesPullingTime="5.087559176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:05.921404961 +0000 UTC m=+72.049491652" watchObservedRunningTime="2026-07-07 02:19:05.922039875 +0000 UTC m=+72.050126568"
Jul 07 02:19:06.878077 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:06.878043    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr" event={"ID":"f0eb21a1-d159-4e50-8f8a-548cca607197","Type":"ContainerStarted","Data":"38686ff80bcce4ba7803384a4abc4de61190a1b7d7ad3322b0c5bdb350d93b81"}
Jul 07 02:19:06.895169 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:06.895112    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tr6tr" podStartSLOduration=35.579901775 podStartE2EDuration="36.895097196s" totalImagesPullingTime="1.315195421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:06.893918137 +0000 UTC m=+73.022004828" watchObservedRunningTime="2026-07-07 02:19:06.895097196 +0000 UTC m=+73.023183887"
Jul 07 02:19:07.824268 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:07.824235    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:19:08.234094 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.234057    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vbq2k"]
Jul 07 02:19:08.240909 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.240881    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.244362 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.244187    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 02:19:08.244362 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.244232    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8twrq\""
Jul 07 02:19:08.245367 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.245344    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 02:19:08.245608 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.245579    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 02:19:08.245801 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.245783    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 02:19:08.245989 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.245973    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 02:19:08.246177 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.246162    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.388778    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcxlj\" (UniqueName: \"kubernetes.io/projected/fd035722-e83d-4655-a185-6491fe845cc0-kube-api-access-xcxlj\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.388835    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-textfile\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.388868    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.388892    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.388924    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.388955    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-wtmp\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.388978    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-root\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.389016    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd035722-e83d-4655-a185-6491fe845cc0-metrics-client-ca\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.389226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.389072    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-sys\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.489879    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcxlj\" (UniqueName: \"kubernetes.io/projected/fd035722-e83d-4655-a185-6491fe845cc0-kube-api-access-xcxlj\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.489943    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-textfile\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.489977    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490001    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490031    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490067    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-wtmp\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490092    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-root\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490131    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd035722-e83d-4655-a185-6491fe845cc0-metrics-client-ca\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490171    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-sys\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490256    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-sys\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.490889    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-textfile\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:19:08.490996    2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:19:08.491053    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls podName:fd035722-e83d-4655-a185-6491fe845cc0 nodeName:}" failed. No retries permitted until 2026-07-07 02:19:08.991032313 +0000 UTC m=+75.119118988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls") pod "node-exporter-vbq2k" (UID: "fd035722-e83d-4655-a185-6491fe845cc0") : secret "node-exporter-tls" not found
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.491627    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-root\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.492106    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.492242    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-wtmp\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.494584 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.492718    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd035722-e83d-4655-a185-6491fe845cc0-metrics-client-ca\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.510273 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.510220    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.514476 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.512962    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcxlj\" (UniqueName: \"kubernetes.io/projected/fd035722-e83d-4655-a185-6491fe845cc0-kube-api-access-xcxlj\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.996074 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:08.995684    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:08.996074 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:19:08.995833    2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 02:19:08.996074 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:19:08.995895    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls podName:fd035722-e83d-4655-a185-6491fe845cc0 nodeName:}" failed. No retries permitted until 2026-07-07 02:19:09.995875757 +0000 UTC m=+76.123962433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls") pod "node-exporter-vbq2k" (UID: "fd035722-e83d-4655-a185-6491fe845cc0") : secret "node-exporter-tls" not found
Jul 07 02:19:09.892960 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:09.892922    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c5f95f59-q8p2f" event={"ID":"9dad02ae-1e12-4b6a-b0c7-5c74d553e612","Type":"ContainerStarted","Data":"1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4"}
Jul 07 02:19:09.911800 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:09.911747    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66c5f95f59-q8p2f" podStartSLOduration=4.74952896 podStartE2EDuration="8.91172941s" totalImagesPullingTime="4.16220045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:09.910186603 +0000 UTC m=+76.038273295" watchObservedRunningTime="2026-07-07 02:19:09.91172941 +0000 UTC m=+76.039816100"
Jul 07 02:19:10.006023 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:10.005981    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:10.008520 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:10.008497    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd035722-e83d-4655-a185-6491fe845cc0-node-exporter-tls\") pod \"node-exporter-vbq2k\" (UID: \"fd035722-e83d-4655-a185-6491fe845cc0\") " pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:10.057705 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:10.057670    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vbq2k"
Jul 07 02:19:10.068200 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:19:10.068174    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd035722_e83d_4655_a185_6491fe845cc0.slice/crio-20fa28fb8f23b3d0443a040a899211945a893afcdb84424a39670587aa8c9c15 WatchSource:0}: Error finding container 20fa28fb8f23b3d0443a040a899211945a893afcdb84424a39670587aa8c9c15: Status 404 returned error can't find the container with id 20fa28fb8f23b3d0443a040a899211945a893afcdb84424a39670587aa8c9c15
Jul 07 02:19:10.898995 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:10.898954    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vbq2k" event={"ID":"fd035722-e83d-4655-a185-6491fe845cc0","Type":"ContainerStarted","Data":"20fa28fb8f23b3d0443a040a899211945a893afcdb84424a39670587aa8c9c15"}
Jul 07 02:19:11.904830 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:11.904791    2585 generic.go:356] "Generic (PLEG): container finished" podID="fd035722-e83d-4655-a185-6491fe845cc0" containerID="224cef91280c36a65ca2893b9f00ec41d5e654fdd81aa31359e4728bf280d283" exitCode=0
Jul 07 02:19:11.905304 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:11.904873    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vbq2k" event={"ID":"fd035722-e83d-4655-a185-6491fe845cc0","Type":"ContainerDied","Data":"224cef91280c36a65ca2893b9f00ec41d5e654fdd81aa31359e4728bf280d283"}
Jul 07 02:19:12.043636 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:12.043596    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:12.043760 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:12.043680    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:12.049828 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:12.049800    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:12.911923 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:12.911870    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vbq2k" event={"ID":"fd035722-e83d-4655-a185-6491fe845cc0","Type":"ContainerStarted","Data":"0375c8b62a153a3731268a6aa1d301316cbe981f72aa09423a9a5f7bea653bce"}
Jul 07 02:19:12.912388 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:12.911953    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vbq2k" event={"ID":"fd035722-e83d-4655-a185-6491fe845cc0","Type":"ContainerStarted","Data":"9f0eb06f1b8470cf52d65e5c398e08c3cd2bebd00163b316fd85d556c47ce5f2"}
Jul 07 02:19:12.917107 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:12.917069    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:12.937314 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:12.937260    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vbq2k" podStartSLOduration=3.7942171890000003 podStartE2EDuration="4.937244988s" totalImagesPullingTime="1.055565916s" totalInitContainerRuntime="87.461883ms" isStatefulPod=true podCreationTimestamp="2026-07-07 02:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:12.935380711 +0000 UTC m=+79.063467402" watchObservedRunningTime="2026-07-07 02:19:12.937244988 +0000 UTC m=+79.065331678"
Jul 07 02:19:19.832419 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:19.832302    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c849fb7f4-f7sxk"
Jul 07 02:19:20.944176 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:20.944129    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-44thl" event={"ID":"0650f4f3-6638-486c-b6a8-92eff0dd11a1","Type":"ContainerStarted","Data":"1caf34c01317216f76ec23508a5d4034c36b7e6900ce9ce8f08aa6e0e3c1f149"}
Jul 07 02:19:20.944679 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:20.944369    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-44thl"
Jul 07 02:19:20.957696 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:20.957665    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-44thl"
Jul 07 02:19:20.965204 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:20.965144    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-44thl" podStartSLOduration=2.086591249 podStartE2EDuration="23.96512742s" totalImagesPullingTime="21.878536171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:20.962877071 +0000 UTC m=+87.090963765" watchObservedRunningTime="2026-07-07 02:19:20.96512742 +0000 UTC m=+87.093214114"
Jul 07 02:19:22.841832 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:22.841789    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph" podUID="ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" containerName="registry" containerID="cri-o://a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202" gracePeriod=30
Jul 07 02:19:23.145862 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.145827    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:19:23.319381 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.319340    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-image-registry-private-configuration\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.319813 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.319788    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.319934 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.319837    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-ca-trust-extracted\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.319934 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.319868    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-bound-sa-token\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.319934 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.319899    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-trusted-ca\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.320310 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.319964    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-installation-pull-secrets\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.320310 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.319989    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npkfs\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-kube-api-access-npkfs\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.320310 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.320015    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-certificates\") pod \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\" (UID: \"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8\") "
Jul 07 02:19:23.320624 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.320595    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:19:23.320972 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.320887    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:19:23.322695 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.322664    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:19:23.323077 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.323031    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:19:23.323595 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.323432    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:19:23.323697 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.323642    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-kube-api-access-npkfs" (OuterVolumeSpecName: "kube-api-access-npkfs") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "kube-api-access-npkfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:19:23.324557 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.324527    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:19:23.332564 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.332531    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" (UID: "ef8961d7-c2fb-4066-ad2b-feb9b153a2b8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:19:23.421626 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421542    2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.421626 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421579    2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-ca-trust-extracted\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.421626 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421595    2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-bound-sa-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.421626 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421611    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-trusted-ca\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.421626 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421625    2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-installation-pull-secrets\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.421952 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421639    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npkfs\" (UniqueName: \"kubernetes.io/projected/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-kube-api-access-npkfs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.421952 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421650    2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-registry-certificates\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.421952 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.421660    2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8-image-registry-private-configuration\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:23.959012 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.958973    2585 generic.go:356] "Generic (PLEG): container finished" podID="ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" containerID="a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202" exitCode=0
Jul 07 02:19:23.959471 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.959019    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph"
Jul 07 02:19:23.959471 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.959068    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph" event={"ID":"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8","Type":"ContainerDied","Data":"a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202"}
Jul 07 02:19:23.959471 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.959130    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f64d5f4b-qxgph" event={"ID":"ef8961d7-c2fb-4066-ad2b-feb9b153a2b8","Type":"ContainerDied","Data":"6a3982abb8a8359fc6f198cf724e7fd96d72f2f81215833eaebb9088f542203e"}
Jul 07 02:19:23.959471 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.959164    2585 scope.go:117] "RemoveContainer" containerID="a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202"
Jul 07 02:19:23.969507 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.969489    2585 scope.go:117] "RemoveContainer" containerID="a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202"
Jul 07 02:19:23.969826 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:19:23.969802    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202\": container with ID starting with a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202 not found: ID does not exist" containerID="a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202"
Jul 07 02:19:23.969939 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.969833    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202"} err="failed to get container status \"a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202\": rpc error: code = NotFound desc = could not find container \"a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202\": container with ID starting with a398e747868b6cc3cc265b75f533e0b70f24e5fdbad9ea3ea8a9364c612bd202 not found: ID does not exist"
Jul 07 02:19:23.983364 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.983329    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77f64d5f4b-qxgph"]
Jul 07 02:19:23.987308 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:23.987279    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77f64d5f4b-qxgph"]
Jul 07 02:19:24.398273 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:24.398194    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" path="/var/lib/kubelet/pods/ef8961d7-c2fb-4066-ad2b-feb9b153a2b8/volumes"
Jul 07 02:19:25.098158 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.098126    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c5f95f59-q8p2f"]
Jul 07 02:19:25.850086 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.850046    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79589f9d77-5c6zq"]
Jul 07 02:19:25.850362 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.850345    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" containerName="registry"
Jul 07 02:19:25.850362 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.850364    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" containerName="registry"
Jul 07 02:19:25.850501 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.850404    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef8961d7-c2fb-4066-ad2b-feb9b153a2b8" containerName="registry"
Jul 07 02:19:25.885289 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.885253    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79589f9d77-5c6zq"]
Jul 07 02:19:25.885525 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.885366    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:25.902700 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:25.902670    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 02:19:26.045894 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.045862    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-oauth-serving-cert\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.046071 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.045911    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-service-ca\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.046071 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.045940    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-586v6\" (UniqueName: \"kubernetes.io/projected/813d04ac-060a-47aa-94cd-672d9d6f7641-kube-api-access-586v6\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.046170 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.046058    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-trusted-ca-bundle\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.046170 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.046098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-console-config\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.046170 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.046121    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-oauth-config\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.046170 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.046154    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-serving-cert\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.147210 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147127    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-trusted-ca-bundle\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.147210 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147176    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-console-config\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.147210 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147205    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-oauth-config\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.147819 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147233    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-serving-cert\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.147819 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147278    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-oauth-serving-cert\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.147819 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147320    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-service-ca\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.147819 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147345    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-586v6\" (UniqueName: \"kubernetes.io/projected/813d04ac-060a-47aa-94cd-672d9d6f7641-kube-api-access-586v6\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.148016 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.147990    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-console-config\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.148280 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.148254    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-trusted-ca-bundle\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.148614 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.148570    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-oauth-serving-cert\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.149114 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.149089    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-service-ca\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.150602 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.150580    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-oauth-config\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.150877 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.150849    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-serving-cert\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.157081 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.157033    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-586v6\" (UniqueName: \"kubernetes.io/projected/813d04ac-060a-47aa-94cd-672d9d6f7641-kube-api-access-586v6\") pod \"console-79589f9d77-5c6zq\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") " pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.196683 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.196646    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:26.379089 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.379050    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79589f9d77-5c6zq"]
Jul 07 02:19:26.383811 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:19:26.383777    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813d04ac_060a_47aa_94cd_672d9d6f7641.slice/crio-472aecd0d271b6aef93eb72b56a0a8c591fcbd2f607ce3994d44a65c54361308 WatchSource:0}: Error finding container 472aecd0d271b6aef93eb72b56a0a8c591fcbd2f607ce3994d44a65c54361308: Status 404 returned error can't find the container with id 472aecd0d271b6aef93eb72b56a0a8c591fcbd2f607ce3994d44a65c54361308
Jul 07 02:19:26.975077 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.975033    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79589f9d77-5c6zq" event={"ID":"813d04ac-060a-47aa-94cd-672d9d6f7641","Type":"ContainerStarted","Data":"c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357"}
Jul 07 02:19:26.975238 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.975110    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79589f9d77-5c6zq" event={"ID":"813d04ac-060a-47aa-94cd-672d9d6f7641","Type":"ContainerStarted","Data":"472aecd0d271b6aef93eb72b56a0a8c591fcbd2f607ce3994d44a65c54361308"}
Jul 07 02:19:26.999867 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:26.999795    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79589f9d77-5c6zq" podStartSLOduration=1.999776292 podStartE2EDuration="1.999776292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:19:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:26.999135653 +0000 UTC m=+93.127222345" watchObservedRunningTime="2026-07-07 02:19:26.999776292 +0000 UTC m=+93.127862985"
Jul 07 02:19:36.197432 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:36.197383    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:36.197897 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:36.197486    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:36.202217 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:36.202196    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:37.014458 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:37.014426    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:19:37.884946 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:37.884912    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bfq5k"
Jul 07 02:19:50.124159 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.124090    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66c5f95f59-q8p2f" podUID="9dad02ae-1e12-4b6a-b0c7-5c74d553e612" containerName="console" containerID="cri-o://1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4" gracePeriod=15
Jul 07 02:19:50.394353 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.394271    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:50.429978 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.429948    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-serving-cert\") pod \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") "
Jul 07 02:19:50.430160 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.429994    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-config\") pod \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") "
Jul 07 02:19:50.430160 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.430046    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-oauth-serving-cert\") pod \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") "
Jul 07 02:19:50.430160 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.430078    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkrjn\" (UniqueName: \"kubernetes.io/projected/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-kube-api-access-nkrjn\") pod \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") "
Jul 07 02:19:50.430160 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.430112    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-oauth-config\") pod \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") "
Jul 07 02:19:50.430350 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.430237    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-service-ca\") pod \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\" (UID: \"9dad02ae-1e12-4b6a-b0c7-5c74d553e612\") "
Jul 07 02:19:50.430573 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.430535    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9dad02ae-1e12-4b6a-b0c7-5c74d553e612" (UID: "9dad02ae-1e12-4b6a-b0c7-5c74d553e612"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:19:50.430573 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.430552    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-config" (OuterVolumeSpecName: "console-config") pod "9dad02ae-1e12-4b6a-b0c7-5c74d553e612" (UID: "9dad02ae-1e12-4b6a-b0c7-5c74d553e612"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:19:50.430724 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.430619    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-service-ca" (OuterVolumeSpecName: "service-ca") pod "9dad02ae-1e12-4b6a-b0c7-5c74d553e612" (UID: "9dad02ae-1e12-4b6a-b0c7-5c74d553e612"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:19:50.432494 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.432462    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9dad02ae-1e12-4b6a-b0c7-5c74d553e612" (UID: "9dad02ae-1e12-4b6a-b0c7-5c74d553e612"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:19:50.432602 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.432505    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-kube-api-access-nkrjn" (OuterVolumeSpecName: "kube-api-access-nkrjn") pod "9dad02ae-1e12-4b6a-b0c7-5c74d553e612" (UID: "9dad02ae-1e12-4b6a-b0c7-5c74d553e612"). InnerVolumeSpecName "kube-api-access-nkrjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:19:50.432602 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.432540    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9dad02ae-1e12-4b6a-b0c7-5c74d553e612" (UID: "9dad02ae-1e12-4b6a-b0c7-5c74d553e612"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:19:50.530925 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.530883    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-service-ca\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:50.530925 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.530917    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-serving-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:50.530925 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.530929    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:50.531152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.530941    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-oauth-serving-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:50.531152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.530953    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkrjn\" (UniqueName: \"kubernetes.io/projected/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-kube-api-access-nkrjn\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:50.531152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:50.530965    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9dad02ae-1e12-4b6a-b0c7-5c74d553e612-console-oauth-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:19:51.060985 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.060952    2585 generic.go:356] "Generic (PLEG): container finished" podID="9dad02ae-1e12-4b6a-b0c7-5c74d553e612" containerID="1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4" exitCode=0
Jul 07 02:19:51.061147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.061011    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c5f95f59-q8p2f"
Jul 07 02:19:51.061147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.061038    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c5f95f59-q8p2f" event={"ID":"9dad02ae-1e12-4b6a-b0c7-5c74d553e612","Type":"ContainerDied","Data":"1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4"}
Jul 07 02:19:51.061147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.061091    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c5f95f59-q8p2f" event={"ID":"9dad02ae-1e12-4b6a-b0c7-5c74d553e612","Type":"ContainerDied","Data":"468a5179ad3cd306ecf4ef9e8d0a83dc9acb6475a67ea68b2f444ffeebc55ec9"}
Jul 07 02:19:51.061147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.061112    2585 scope.go:117] "RemoveContainer" containerID="1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4"
Jul 07 02:19:51.069663 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.069646    2585 scope.go:117] "RemoveContainer" containerID="1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4"
Jul 07 02:19:51.069926 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:19:51.069906    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4\": container with ID starting with 1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4 not found: ID does not exist" containerID="1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4"
Jul 07 02:19:51.069988 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.069938    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4"} err="failed to get container status \"1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4\": rpc error: code = NotFound desc = could not find container \"1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4\": container with ID starting with 1bbbcaa20bd943fafa0b9ee87d218c31f5aae0ef8eea034a1e9b00c116abd9c4 not found: ID does not exist"
Jul 07 02:19:51.081610 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.081590    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c5f95f59-q8p2f"]
Jul 07 02:19:51.085919 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.085902    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66c5f95f59-q8p2f"]
Jul 07 02:19:51.601973 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:51.601944    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kh65r_5244c428-40dd-4db7-acb9-5723befa306b/serve-healthcheck-canary/0.log"
Jul 07 02:19:52.396229 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:19:52.396196    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dad02ae-1e12-4b6a-b0c7-5c74d553e612" path="/var/lib/kubelet/pods/9dad02ae-1e12-4b6a-b0c7-5c74d553e612/volumes"
Jul 07 02:20:52.591091 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.591013    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d54766466-mwsl8"]
Jul 07 02:20:52.591550 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.591276    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dad02ae-1e12-4b6a-b0c7-5c74d553e612" containerName="console"
Jul 07 02:20:52.591550 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.591288    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dad02ae-1e12-4b6a-b0c7-5c74d553e612" containerName="console"
Jul 07 02:20:52.591550 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.591338    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dad02ae-1e12-4b6a-b0c7-5c74d553e612" containerName="console"
Jul 07 02:20:52.593024 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.593006    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.609230 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.609196    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d54766466-mwsl8"]
Jul 07 02:20:52.683144 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.683105    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-trusted-ca-bundle\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.683144 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.683140    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pghqr\" (UniqueName: \"kubernetes.io/projected/79cdf817-f691-458b-83c6-b628d6e1295d-kube-api-access-pghqr\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.683352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.683163    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-serving-cert\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.683352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.683182    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-oauth-config\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.683352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.683225    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-oauth-serving-cert\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.683352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.683278    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-console-config\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.683352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.683299    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-service-ca\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.784365 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.784327    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-console-config\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.784365 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.784366    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-service-ca\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.784611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.784412    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-trusted-ca-bundle\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.784611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.784430    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pghqr\" (UniqueName: \"kubernetes.io/projected/79cdf817-f691-458b-83c6-b628d6e1295d-kube-api-access-pghqr\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.784611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.784593    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-serving-cert\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.784793 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.784648    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-oauth-config\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.784793 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.784699    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-oauth-serving-cert\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.785192 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.785162    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-service-ca\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.785285 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.785177    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-console-config\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.785372 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.785353    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-oauth-serving-cert\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.785854 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.785832    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-trusted-ca-bundle\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.787301 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.787275    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-oauth-config\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.787381 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.787362    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-serving-cert\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.793330 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.793294    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pghqr\" (UniqueName: \"kubernetes.io/projected/79cdf817-f691-458b-83c6-b628d6e1295d-kube-api-access-pghqr\") pod \"console-7d54766466-mwsl8\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") " pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:52.902203 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:52.902111    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:20:53.053753 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:53.053603    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d54766466-mwsl8"]
Jul 07 02:20:53.056649 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:20:53.056612    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79cdf817_f691_458b_83c6_b628d6e1295d.slice/crio-957937c43eb8dafa276cc3c84a7f4e263814a31da4a2fbdea9a3608858e47c80 WatchSource:0}: Error finding container 957937c43eb8dafa276cc3c84a7f4e263814a31da4a2fbdea9a3608858e47c80: Status 404 returned error can't find the container with id 957937c43eb8dafa276cc3c84a7f4e263814a31da4a2fbdea9a3608858e47c80
Jul 07 02:20:53.267629 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:53.267592    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d54766466-mwsl8" event={"ID":"79cdf817-f691-458b-83c6-b628d6e1295d","Type":"ContainerStarted","Data":"78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1"}
Jul 07 02:20:53.267802 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:53.267643    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d54766466-mwsl8" event={"ID":"79cdf817-f691-458b-83c6-b628d6e1295d","Type":"ContainerStarted","Data":"957937c43eb8dafa276cc3c84a7f4e263814a31da4a2fbdea9a3608858e47c80"}
Jul 07 02:20:53.286387 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:20:53.286330    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d54766466-mwsl8" podStartSLOduration=1.286277269 podStartE2EDuration="1.286277269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:53.285954182 +0000 UTC m=+179.414040883" watchObservedRunningTime="2026-07-07 02:20:53.286277269 +0000 UTC m=+179.414363960"
Jul 07 02:21:02.902589 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:02.902543    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:21:02.903039 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:02.902618    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:21:02.907251 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:02.907228    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:21:03.304075 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:03.304041    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:21:03.374429 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:03.374386    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79589f9d77-5c6zq"]
Jul 07 02:21:28.397121 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.397066    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-79589f9d77-5c6zq" podUID="813d04ac-060a-47aa-94cd-672d9d6f7641" containerName="console" containerID="cri-o://c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357" gracePeriod=15
Jul 07 02:21:28.635781 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.635755    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:21:28.731392 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731355    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-586v6\" (UniqueName: \"kubernetes.io/projected/813d04ac-060a-47aa-94cd-672d9d6f7641-kube-api-access-586v6\") pod \"813d04ac-060a-47aa-94cd-672d9d6f7641\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") "
Jul 07 02:21:28.731606 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731404    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-console-config\") pod \"813d04ac-060a-47aa-94cd-672d9d6f7641\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") "
Jul 07 02:21:28.731606 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731429    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-service-ca\") pod \"813d04ac-060a-47aa-94cd-672d9d6f7641\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") "
Jul 07 02:21:28.731606 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731479    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-oauth-config\") pod \"813d04ac-060a-47aa-94cd-672d9d6f7641\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") "
Jul 07 02:21:28.731606 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731538    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-trusted-ca-bundle\") pod \"813d04ac-060a-47aa-94cd-672d9d6f7641\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") "
Jul 07 02:21:28.731795 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731683    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-oauth-serving-cert\") pod \"813d04ac-060a-47aa-94cd-672d9d6f7641\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") "
Jul 07 02:21:28.731795 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731727    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-serving-cert\") pod \"813d04ac-060a-47aa-94cd-672d9d6f7641\" (UID: \"813d04ac-060a-47aa-94cd-672d9d6f7641\") "
Jul 07 02:21:28.731896 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731842    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-console-config" (OuterVolumeSpecName: "console-config") pod "813d04ac-060a-47aa-94cd-672d9d6f7641" (UID: "813d04ac-060a-47aa-94cd-672d9d6f7641"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:21:28.731952 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.731941    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-console-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:21:28.732044 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.732013    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-service-ca" (OuterVolumeSpecName: "service-ca") pod "813d04ac-060a-47aa-94cd-672d9d6f7641" (UID: "813d04ac-060a-47aa-94cd-672d9d6f7641"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:21:28.732111 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.732089    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "813d04ac-060a-47aa-94cd-672d9d6f7641" (UID: "813d04ac-060a-47aa-94cd-672d9d6f7641"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:21:28.732176 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.732148    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "813d04ac-060a-47aa-94cd-672d9d6f7641" (UID: "813d04ac-060a-47aa-94cd-672d9d6f7641"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:21:28.733898 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.733874    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "813d04ac-060a-47aa-94cd-672d9d6f7641" (UID: "813d04ac-060a-47aa-94cd-672d9d6f7641"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:21:28.733992 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.733934    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "813d04ac-060a-47aa-94cd-672d9d6f7641" (UID: "813d04ac-060a-47aa-94cd-672d9d6f7641"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:21:28.733992 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.733949    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813d04ac-060a-47aa-94cd-672d9d6f7641-kube-api-access-586v6" (OuterVolumeSpecName: "kube-api-access-586v6") pod "813d04ac-060a-47aa-94cd-672d9d6f7641" (UID: "813d04ac-060a-47aa-94cd-672d9d6f7641"). InnerVolumeSpecName "kube-api-access-586v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:21:28.832826 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.832794    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-586v6\" (UniqueName: \"kubernetes.io/projected/813d04ac-060a-47aa-94cd-672d9d6f7641-kube-api-access-586v6\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:21:28.832826 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.832822    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-service-ca\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:21:28.832826 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.832830    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-oauth-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:21:28.833044 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.832839    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-trusted-ca-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:21:28.833044 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.832848    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/813d04ac-060a-47aa-94cd-672d9d6f7641-oauth-serving-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:21:28.833044 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:28.832856    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/813d04ac-060a-47aa-94cd-672d9d6f7641-console-serving-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:21:29.387873 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.387836    2585 generic.go:356] "Generic (PLEG): container finished" podID="813d04ac-060a-47aa-94cd-672d9d6f7641" containerID="c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357" exitCode=0
Jul 07 02:21:29.388051 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.387888    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79589f9d77-5c6zq"
Jul 07 02:21:29.388051 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.387901    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79589f9d77-5c6zq" event={"ID":"813d04ac-060a-47aa-94cd-672d9d6f7641","Type":"ContainerDied","Data":"c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357"}
Jul 07 02:21:29.388051 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.387953    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79589f9d77-5c6zq" event={"ID":"813d04ac-060a-47aa-94cd-672d9d6f7641","Type":"ContainerDied","Data":"472aecd0d271b6aef93eb72b56a0a8c591fcbd2f607ce3994d44a65c54361308"}
Jul 07 02:21:29.388051 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.387981    2585 scope.go:117] "RemoveContainer" containerID="c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357"
Jul 07 02:21:29.395964 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.395944    2585 scope.go:117] "RemoveContainer" containerID="c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357"
Jul 07 02:21:29.396219 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:21:29.396201    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357\": container with ID starting with c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357 not found: ID does not exist" containerID="c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357"
Jul 07 02:21:29.396268 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.396226    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357"} err="failed to get container status \"c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357\": rpc error: code = NotFound desc = could not find container \"c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357\": container with ID starting with c0f5cb1ed3e7cad7da2511b20053e3c53d959fa0525dea27f272c5fb5a395357 not found: ID does not exist"
Jul 07 02:21:29.409332 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.409306    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79589f9d77-5c6zq"]
Jul 07 02:21:29.415717 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:29.415698    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79589f9d77-5c6zq"]
Jul 07 02:21:30.395341 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:30.395306    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813d04ac-060a-47aa-94cd-672d9d6f7641" path="/var/lib/kubelet/pods/813d04ac-060a-47aa-94cd-672d9d6f7641/volumes"
Jul 07 02:21:36.379793 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.379758    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"]
Jul 07 02:21:36.380161 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.379996    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813d04ac-060a-47aa-94cd-672d9d6f7641" containerName="console"
Jul 07 02:21:36.380161 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.380008    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="813d04ac-060a-47aa-94cd-672d9d6f7641" containerName="console"
Jul 07 02:21:36.380161 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.380057    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="813d04ac-060a-47aa-94cd-672d9d6f7641" containerName="console"
Jul 07 02:21:36.382559 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.382542    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.387129 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.387105    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4xtpp\""
Jul 07 02:21:36.387129 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.387121    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 02:21:36.387282 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.387105    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 02:21:36.387282 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.387106    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 02:21:36.387282 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.387165    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 02:21:36.391880 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.391862    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"]
Jul 07 02:21:36.484663 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.484631    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f43574d4-da8b-4b43-ba74-1aaa3d38a578-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cf459966b-5c7xc\" (UID: \"f43574d4-da8b-4b43-ba74-1aaa3d38a578\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.484815 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.484694    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5962\" (UniqueName: \"kubernetes.io/projected/f43574d4-da8b-4b43-ba74-1aaa3d38a578-kube-api-access-c5962\") pod \"managed-serviceaccount-addon-agent-6cf459966b-5c7xc\" (UID: \"f43574d4-da8b-4b43-ba74-1aaa3d38a578\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.585748 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.585717    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5962\" (UniqueName: \"kubernetes.io/projected/f43574d4-da8b-4b43-ba74-1aaa3d38a578-kube-api-access-c5962\") pod \"managed-serviceaccount-addon-agent-6cf459966b-5c7xc\" (UID: \"f43574d4-da8b-4b43-ba74-1aaa3d38a578\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.585930 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.585762    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f43574d4-da8b-4b43-ba74-1aaa3d38a578-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cf459966b-5c7xc\" (UID: \"f43574d4-da8b-4b43-ba74-1aaa3d38a578\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.588384 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.588350    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f43574d4-da8b-4b43-ba74-1aaa3d38a578-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cf459966b-5c7xc\" (UID: \"f43574d4-da8b-4b43-ba74-1aaa3d38a578\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.594723 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.594700    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5962\" (UniqueName: \"kubernetes.io/projected/f43574d4-da8b-4b43-ba74-1aaa3d38a578-kube-api-access-c5962\") pod \"managed-serviceaccount-addon-agent-6cf459966b-5c7xc\" (UID: \"f43574d4-da8b-4b43-ba74-1aaa3d38a578\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.700513 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.700480    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"
Jul 07 02:21:36.825269 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:36.825236    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc"]
Jul 07 02:21:36.828014 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:21:36.827987    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf43574d4_da8b_4b43_ba74_1aaa3d38a578.slice/crio-802b79891e96a5bba7919a240e241a6dc687b36769f361bef1a677952d7917ce WatchSource:0}: Error finding container 802b79891e96a5bba7919a240e241a6dc687b36769f361bef1a677952d7917ce: Status 404 returned error can't find the container with id 802b79891e96a5bba7919a240e241a6dc687b36769f361bef1a677952d7917ce
Jul 07 02:21:37.415091 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:37.415050    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc" event={"ID":"f43574d4-da8b-4b43-ba74-1aaa3d38a578","Type":"ContainerStarted","Data":"802b79891e96a5bba7919a240e241a6dc687b36769f361bef1a677952d7917ce"}
Jul 07 02:21:40.431725 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:40.431686    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc" event={"ID":"f43574d4-da8b-4b43-ba74-1aaa3d38a578","Type":"ContainerStarted","Data":"aae5a13b2b2ed0faad71a8a71f50c25217e56675636a3775bf33b8e437f94edc"}
Jul 07 02:21:40.449761 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:21:40.449709    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cf459966b-5c7xc" podStartSLOduration=1.807654623 podStartE2EDuration="4.449694259s" totalImagesPullingTime="2.642039636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:40.449414317 +0000 UTC m=+226.577501008" watchObservedRunningTime="2026-07-07 02:21:40.449694259 +0000 UTC m=+226.577780949"
Jul 07 02:22:21.883963 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:21.883886    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"]
Jul 07 02:22:21.887238 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:21.887216    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:21.890487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:21.890463    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 02:22:21.890608 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:21.890490    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d8k4f\""
Jul 07 02:22:21.891745 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:21.891726    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 02:22:21.896090 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:21.896066    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"]
Jul 07 02:22:22.028007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.027962    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.028177 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.028026    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.028177 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.028053    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7chd\" (UniqueName: \"kubernetes.io/projected/9108d318-ebb5-491c-9464-573e15e5d89a-kube-api-access-s7chd\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.129299 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.129246    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.129484 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.129313    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7chd\" (UniqueName: \"kubernetes.io/projected/9108d318-ebb5-491c-9464-573e15e5d89a-kube-api-access-s7chd\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.129484 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.129404    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.129701 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.129683    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.129798 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.129779    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.138938 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.138884    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7chd\" (UniqueName: \"kubernetes.io/projected/9108d318-ebb5-491c-9464-573e15e5d89a-kube-api-access-s7chd\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.196901 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.196866    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:22.331218 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.331188    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"]
Jul 07 02:22:22.333897 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:22:22.333863    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9108d318_ebb5_491c_9464_573e15e5d89a.slice/crio-e2b21206bc3d608f3740da6d1055ab0a54d33847474dc8bdac370fa4f1373fe5 WatchSource:0}: Error finding container e2b21206bc3d608f3740da6d1055ab0a54d33847474dc8bdac370fa4f1373fe5: Status 404 returned error can't find the container with id e2b21206bc3d608f3740da6d1055ab0a54d33847474dc8bdac370fa4f1373fe5
Jul 07 02:22:22.571630 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:22.571592    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87" event={"ID":"9108d318-ebb5-491c-9464-573e15e5d89a","Type":"ContainerStarted","Data":"e2b21206bc3d608f3740da6d1055ab0a54d33847474dc8bdac370fa4f1373fe5"}
Jul 07 02:22:30.603732 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:30.603694    2585 generic.go:356] "Generic (PLEG): container finished" podID="9108d318-ebb5-491c-9464-573e15e5d89a" containerID="97cf5c1b6e3b3d13be01993ee36f5fa428ae3c22790361271b0dbdcb8ed931bd" exitCode=0
Jul 07 02:22:30.604167 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:30.603791    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87" event={"ID":"9108d318-ebb5-491c-9464-573e15e5d89a","Type":"ContainerDied","Data":"97cf5c1b6e3b3d13be01993ee36f5fa428ae3c22790361271b0dbdcb8ed931bd"}
Jul 07 02:22:33.616706 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:33.616673    2585 generic.go:356] "Generic (PLEG): container finished" podID="9108d318-ebb5-491c-9464-573e15e5d89a" containerID="df5d45c6741c1ad3943d0fa637880aa8a489a515c319eca24133b00af57938d0" exitCode=0
Jul 07 02:22:33.617142 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:33.616765    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87" event={"ID":"9108d318-ebb5-491c-9464-573e15e5d89a","Type":"ContainerDied","Data":"df5d45c6741c1ad3943d0fa637880aa8a489a515c319eca24133b00af57938d0"}
Jul 07 02:22:40.649007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:40.648968    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87" event={"ID":"9108d318-ebb5-491c-9464-573e15e5d89a","Type":"ContainerStarted","Data":"1348e04a69aa20be43a80bf03143d9bc4736c5db3f65ad74be158794ec03546b"}
Jul 07 02:22:40.668935 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:40.668885    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87" podStartSLOduration=2.953146271 podStartE2EDuration="19.668868452s" totalImagesPullingTime="16.715722181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:40.668042534 +0000 UTC m=+286.796129226" watchObservedRunningTime="2026-07-07 02:22:40.668868452 +0000 UTC m=+286.796955143"
Jul 07 02:22:41.654740 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:41.654703    2585 generic.go:356] "Generic (PLEG): container finished" podID="9108d318-ebb5-491c-9464-573e15e5d89a" containerID="1348e04a69aa20be43a80bf03143d9bc4736c5db3f65ad74be158794ec03546b" exitCode=0
Jul 07 02:22:41.655126 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:41.654747    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87" event={"ID":"9108d318-ebb5-491c-9464-573e15e5d89a","Type":"ContainerDied","Data":"1348e04a69aa20be43a80bf03143d9bc4736c5db3f65ad74be158794ec03546b"}
Jul 07 02:22:42.778398 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.778371    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:42.790557 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.790523    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-bundle\") pod \"9108d318-ebb5-491c-9464-573e15e5d89a\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") "
Jul 07 02:22:42.790709 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.790619    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7chd\" (UniqueName: \"kubernetes.io/projected/9108d318-ebb5-491c-9464-573e15e5d89a-kube-api-access-s7chd\") pod \"9108d318-ebb5-491c-9464-573e15e5d89a\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") "
Jul 07 02:22:42.790709 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.790651    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-util\") pod \"9108d318-ebb5-491c-9464-573e15e5d89a\" (UID: \"9108d318-ebb5-491c-9464-573e15e5d89a\") "
Jul 07 02:22:42.791298 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.791272    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-bundle" (OuterVolumeSpecName: "bundle") pod "9108d318-ebb5-491c-9464-573e15e5d89a" (UID: "9108d318-ebb5-491c-9464-573e15e5d89a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:22:42.792856 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.792835    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9108d318-ebb5-491c-9464-573e15e5d89a-kube-api-access-s7chd" (OuterVolumeSpecName: "kube-api-access-s7chd") pod "9108d318-ebb5-491c-9464-573e15e5d89a" (UID: "9108d318-ebb5-491c-9464-573e15e5d89a"). InnerVolumeSpecName "kube-api-access-s7chd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:22:42.797371 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.797343    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-util" (OuterVolumeSpecName: "util") pod "9108d318-ebb5-491c-9464-573e15e5d89a" (UID: "9108d318-ebb5-491c-9464-573e15e5d89a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 02:22:42.891141 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.891111    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:22:42.891141 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.891138    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7chd\" (UniqueName: \"kubernetes.io/projected/9108d318-ebb5-491c-9464-573e15e5d89a-kube-api-access-s7chd\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:22:42.891141 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:42.891148    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9108d318-ebb5-491c-9464-573e15e5d89a-util\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:22:43.663343 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:43.663311    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87" event={"ID":"9108d318-ebb5-491c-9464-573e15e5d89a","Type":"ContainerDied","Data":"e2b21206bc3d608f3740da6d1055ab0a54d33847474dc8bdac370fa4f1373fe5"}
Jul 07 02:22:43.663343 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:43.663331    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589g5n87"
Jul 07 02:22:43.663574 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:43.663358    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2b21206bc3d608f3740da6d1055ab0a54d33847474dc8bdac370fa4f1373fe5"
Jul 07 02:22:49.134705 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.134668    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"]
Jul 07 02:22:49.135152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.134908    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9108d318-ebb5-491c-9464-573e15e5d89a" containerName="util"
Jul 07 02:22:49.135152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.134918    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9108d318-ebb5-491c-9464-573e15e5d89a" containerName="util"
Jul 07 02:22:49.135152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.134933    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9108d318-ebb5-491c-9464-573e15e5d89a" containerName="pull"
Jul 07 02:22:49.135152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.134939    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9108d318-ebb5-491c-9464-573e15e5d89a" containerName="pull"
Jul 07 02:22:49.135152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.134950    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9108d318-ebb5-491c-9464-573e15e5d89a" containerName="extract"
Jul 07 02:22:49.135152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.134955    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9108d318-ebb5-491c-9464-573e15e5d89a" containerName="extract"
Jul 07 02:22:49.135152 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.135000    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9108d318-ebb5-491c-9464-573e15e5d89a" containerName="extract"
Jul 07 02:22:49.138997 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.138979    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.142560 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.142528    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 07 02:22:49.142724 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.142534    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 07 02:22:49.142884 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.142867    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-dmbds\""
Jul 07 02:22:49.142974 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.142958    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 07 02:22:49.157196 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.157168    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"]
Jul 07 02:22:49.239786 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.239748    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c7328cbe-9c21-4073-ba8a-98bfa62d50b3-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p\" (UID: \"c7328cbe-9c21-4073-ba8a-98bfa62d50b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.239954 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.239797    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4kmj\" (UniqueName: \"kubernetes.io/projected/c7328cbe-9c21-4073-ba8a-98bfa62d50b3-kube-api-access-d4kmj\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p\" (UID: \"c7328cbe-9c21-4073-ba8a-98bfa62d50b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.341023 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.340977    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c7328cbe-9c21-4073-ba8a-98bfa62d50b3-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p\" (UID: \"c7328cbe-9c21-4073-ba8a-98bfa62d50b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.341163 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.341038    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4kmj\" (UniqueName: \"kubernetes.io/projected/c7328cbe-9c21-4073-ba8a-98bfa62d50b3-kube-api-access-d4kmj\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p\" (UID: \"c7328cbe-9c21-4073-ba8a-98bfa62d50b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.343526 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.343505    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c7328cbe-9c21-4073-ba8a-98bfa62d50b3-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p\" (UID: \"c7328cbe-9c21-4073-ba8a-98bfa62d50b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.350635 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.350609    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4kmj\" (UniqueName: \"kubernetes.io/projected/c7328cbe-9c21-4073-ba8a-98bfa62d50b3-kube-api-access-d4kmj\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p\" (UID: \"c7328cbe-9c21-4073-ba8a-98bfa62d50b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.449211 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.449175    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:49.585939 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.585906    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"]
Jul 07 02:22:49.589514 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:22:49.589488    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7328cbe_9c21_4073_ba8a_98bfa62d50b3.slice/crio-a7a58026dc9391d4cd77d3d62ad49521da6ee0203025e539d04d78d7dbbce042 WatchSource:0}: Error finding container a7a58026dc9391d4cd77d3d62ad49521da6ee0203025e539d04d78d7dbbce042: Status 404 returned error can't find the container with id a7a58026dc9391d4cd77d3d62ad49521da6ee0203025e539d04d78d7dbbce042
Jul 07 02:22:49.685832 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:49.685797    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p" event={"ID":"c7328cbe-9c21-4073-ba8a-98bfa62d50b3","Type":"ContainerStarted","Data":"a7a58026dc9391d4cd77d3d62ad49521da6ee0203025e539d04d78d7dbbce042"}
Jul 07 02:22:52.698494 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.698436    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p" event={"ID":"c7328cbe-9c21-4073-ba8a-98bfa62d50b3","Type":"ContainerStarted","Data":"f7f8ec5a87b2aa153cd1baf9082fadf149b20d6ba49f928664fabf9fa1a823c2"}
Jul 07 02:22:52.698973 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.698592    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:22:52.721618 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.721559    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p" podStartSLOduration=1.110333061 podStartE2EDuration="3.721541133s" totalImagesPullingTime="2.611208072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:52.718871873 +0000 UTC m=+298.846958564" watchObservedRunningTime="2026-07-07 02:22:52.721541133 +0000 UTC m=+298.849627823"
Jul 07 02:22:52.911320 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.911292    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-tjrqh"]
Jul 07 02:22:52.913301 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.913285    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:52.916261 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.916239    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 07 02:22:52.916516 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.916500    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 07 02:22:52.916516 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.916508    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-pczq7\""
Jul 07 02:22:52.925732 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.925709    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-tjrqh"]
Jul 07 02:22:52.970037 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.969937    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ph2k\" (UniqueName: \"kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-kube-api-access-5ph2k\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:52.970037 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.969997    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-cabundle0\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:52.970228 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:52.970099    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-certificates\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.071374 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.071329    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-cabundle0\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.071576 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.071423    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-certificates\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.071576 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.071476    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ph2k\" (UniqueName: \"kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-kube-api-access-5ph2k\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.071857 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.071831    2585 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 07 02:22:53.071973 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.071863    2585 secret.go:281] references non-existent secret key: ca.crt
Jul 07 02:22:53.071973 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.071874    2585 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 07 02:22:53.071973 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.071902    2585 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-tjrqh: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 07 02:22:53.071973 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.071966    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-certificates podName:c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d nodeName:}" failed. No retries permitted until 2026-07-07 02:22:53.571948427 +0000 UTC m=+299.700035116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-certificates") pod "keda-operator-68bff59c-tjrqh" (UID: "c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 07 02:22:53.072185 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.072160    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-cabundle0\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.081780 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.081757    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ph2k\" (UniqueName: \"kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-kube-api-access-5ph2k\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.265575 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.265490    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"]
Jul 07 02:22:53.267930 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.267911    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.270861 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.270842    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 07 02:22:53.279262 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.279239    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"]
Jul 07 02:22:53.374659 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.374618    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.374659 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.374660    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgzks\" (UniqueName: \"kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-kube-api-access-kgzks\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.374892 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.374680    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-certificates\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.475809 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.475778    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.475976 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.475815    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgzks\" (UniqueName: \"kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-kube-api-access-kgzks\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.475976 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.475846    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-certificates\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.475976 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.475956    2585 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jul 07 02:22:53.475976 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.475977    2585 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8: secret "keda-metrics-apiserver-certs" not found
Jul 07 02:22:53.476127 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.476055    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-certificates podName:d385ca00-0c98-4100-9b58-9abfdb0f0ca7 nodeName:}" failed. No retries permitted until 2026-07-07 02:22:53.976032495 +0000 UTC m=+300.104119179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-certificates") pod "keda-metrics-apiserver-559f75f947-7jbr8" (UID: "d385ca00-0c98-4100-9b58-9abfdb0f0ca7") : secret "keda-metrics-apiserver-certs" not found
Jul 07 02:22:53.476225 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.476206    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.480459 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.480424    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-8vrzp"]
Jul 07 02:22:53.482619 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.482604    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:53.485920 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.485897    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 07 02:22:53.492895 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.492874    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-8vrzp"]
Jul 07 02:22:53.496078 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.496057    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgzks\" (UniqueName: \"kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-kube-api-access-kgzks\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.577129 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.577033    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-certificates\") pod \"keda-admission-5fbd5c4644-8vrzp\" (UID: \"26edd6c6-af10-41d6-a3d8-3f327ba66f8b\") " pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:53.577129 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.577083    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzrsn\" (UniqueName: \"kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-kube-api-access-jzrsn\") pod \"keda-admission-5fbd5c4644-8vrzp\" (UID: \"26edd6c6-af10-41d6-a3d8-3f327ba66f8b\") " pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:53.577355 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.577134    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-certificates\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.579941 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.579905    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d-certificates\") pod \"keda-operator-68bff59c-tjrqh\" (UID: \"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d\") " pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.677663 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.677620    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzrsn\" (UniqueName: \"kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-kube-api-access-jzrsn\") pod \"keda-admission-5fbd5c4644-8vrzp\" (UID: \"26edd6c6-af10-41d6-a3d8-3f327ba66f8b\") " pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:53.677836 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.677761    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-certificates\") pod \"keda-admission-5fbd5c4644-8vrzp\" (UID: \"26edd6c6-af10-41d6-a3d8-3f327ba66f8b\") " pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:53.677924 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.677873    2585 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 07 02:22:53.677924 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.677896    2585 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-8vrzp: secret "keda-admission-webhooks-certs" not found
Jul 07 02:22:53.678033 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:22:53.677978    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-certificates podName:26edd6c6-af10-41d6-a3d8-3f327ba66f8b nodeName:}" failed. No retries permitted until 2026-07-07 02:22:54.177945221 +0000 UTC m=+300.306031909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-certificates") pod "keda-admission-5fbd5c4644-8vrzp" (UID: "26edd6c6-af10-41d6-a3d8-3f327ba66f8b") : secret "keda-admission-webhooks-certs" not found
Jul 07 02:22:53.688955 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.688921    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzrsn\" (UniqueName: \"kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-kube-api-access-jzrsn\") pod \"keda-admission-5fbd5c4644-8vrzp\" (UID: \"26edd6c6-af10-41d6-a3d8-3f327ba66f8b\") " pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:53.822533 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.822498    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:53.955103 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.955074    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-tjrqh"]
Jul 07 02:22:53.957095 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:22:53.957065    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f24ded_0f6a_4dfb_ac1b_0d49ae38fe3d.slice/crio-9a70107916ded8da52b27606dcb833bf0b268054e50f74fe667f138601a368cf WatchSource:0}: Error finding container 9a70107916ded8da52b27606dcb833bf0b268054e50f74fe667f138601a368cf: Status 404 returned error can't find the container with id 9a70107916ded8da52b27606dcb833bf0b268054e50f74fe667f138601a368cf
Jul 07 02:22:53.979342 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.979318    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-certificates\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:53.981810 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:53.981791    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d385ca00-0c98-4100-9b58-9abfdb0f0ca7-certificates\") pod \"keda-metrics-apiserver-559f75f947-7jbr8\" (UID: \"d385ca00-0c98-4100-9b58-9abfdb0f0ca7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:54.178826 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.178743    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:54.181021 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.180997    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-certificates\") pod \"keda-admission-5fbd5c4644-8vrzp\" (UID: \"26edd6c6-af10-41d6-a3d8-3f327ba66f8b\") " pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:54.183635 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.183609    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26edd6c6-af10-41d6-a3d8-3f327ba66f8b-certificates\") pod \"keda-admission-5fbd5c4644-8vrzp\" (UID: \"26edd6c6-af10-41d6-a3d8-3f327ba66f8b\") " pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:54.282867 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.282839    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:22:54.283050 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.282851    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:22:54.289637 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.289614    2585 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 02:22:54.334963 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.323827    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"]
Jul 07 02:22:54.334963 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:22:54.326095    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd385ca00_0c98_4100_9b58_9abfdb0f0ca7.slice/crio-aeac564858bfe78fa199a14f25a48a6dffdac475676da40a27511631ffc2bdb8 WatchSource:0}: Error finding container aeac564858bfe78fa199a14f25a48a6dffdac475676da40a27511631ffc2bdb8: Status 404 returned error can't find the container with id aeac564858bfe78fa199a14f25a48a6dffdac475676da40a27511631ffc2bdb8
Jul 07 02:22:54.392150 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.392119    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:54.542602 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.542490    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-8vrzp"]
Jul 07 02:22:54.544939 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:22:54.544896    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26edd6c6_af10_41d6_a3d8_3f327ba66f8b.slice/crio-c30ce88d998f0924786d0db8959f9d2902f41abd6435322544e4f4f299fb0682 WatchSource:0}: Error finding container c30ce88d998f0924786d0db8959f9d2902f41abd6435322544e4f4f299fb0682: Status 404 returned error can't find the container with id c30ce88d998f0924786d0db8959f9d2902f41abd6435322544e4f4f299fb0682
Jul 07 02:22:54.708424 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.708337    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8" event={"ID":"d385ca00-0c98-4100-9b58-9abfdb0f0ca7","Type":"ContainerStarted","Data":"aeac564858bfe78fa199a14f25a48a6dffdac475676da40a27511631ffc2bdb8"}
Jul 07 02:22:54.709721 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.709677    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp" event={"ID":"26edd6c6-af10-41d6-a3d8-3f327ba66f8b","Type":"ContainerStarted","Data":"c30ce88d998f0924786d0db8959f9d2902f41abd6435322544e4f4f299fb0682"}
Jul 07 02:22:54.710931 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:54.710907    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-tjrqh" event={"ID":"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d","Type":"ContainerStarted","Data":"9a70107916ded8da52b27606dcb833bf0b268054e50f74fe667f138601a368cf"}
Jul 07 02:22:58.729660 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.729619    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp" event={"ID":"26edd6c6-af10-41d6-a3d8-3f327ba66f8b","Type":"ContainerStarted","Data":"9af600ff84fa3dadd0f91bb6be15c01d947bdc8e4e6e6ea465d21c3d57bdf6af"}
Jul 07 02:22:58.730102 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.729721    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:22:58.731089 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.731063    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-tjrqh" event={"ID":"c3f24ded-0f6a-4dfb-ac1b-0d49ae38fe3d","Type":"ContainerStarted","Data":"16e1a2bae5158d9cc1521b5e69b44e84f314a96212150161a858b1dca600f684"}
Jul 07 02:22:58.731242 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.731119    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:22:58.732368 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.732348    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8" event={"ID":"d385ca00-0c98-4100-9b58-9abfdb0f0ca7","Type":"ContainerStarted","Data":"546fb40e0d20689270f7b3a43d751cc4fc7e200ecb6cc3670d257b93cae8ab75"}
Jul 07 02:22:58.732505 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.732492    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:22:58.747917 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.747871    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp" podStartSLOduration=2.419581605 podStartE2EDuration="5.747858036s" totalImagesPullingTime="3.328276431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:58.746337964 +0000 UTC m=+304.874424633" watchObservedRunningTime="2026-07-07 02:22:58.747858036 +0000 UTC m=+304.875944726"
Jul 07 02:22:58.764340 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.764295    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-tjrqh" podStartSLOduration=2.8380912990000002 podStartE2EDuration="6.764282s" totalImagesPullingTime="3.926190701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:58.762881022 +0000 UTC m=+304.890967715" watchObservedRunningTime="2026-07-07 02:22:58.764282 +0000 UTC m=+304.892368691"
Jul 07 02:22:58.779352 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:22:58.779311    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8" podStartSLOduration=2.22221022 podStartE2EDuration="5.779296001s" totalImagesPullingTime="3.557085781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:58.779148228 +0000 UTC m=+304.907234914" watchObservedRunningTime="2026-07-07 02:22:58.779296001 +0000 UTC m=+304.907382689"
Jul 07 02:23:09.741335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:23:09.741307    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-7jbr8"
Jul 07 02:23:13.705782 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:23:13.705752    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-gtv7p"
Jul 07 02:23:19.739069 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:23:19.739036    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-tjrqh"
Jul 07 02:23:19.739494 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:23:19.739466    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-8vrzp"
Jul 07 02:24:01.056005 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.055925    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-95xxp"]
Jul 07 02:24:01.058726 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.058705    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.062511 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.062491    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 07 02:24:01.062627 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.062579    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 07 02:24:01.064118 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.064099    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-l9n6t\""
Jul 07 02:24:01.064240 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.064118    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 07 02:24:01.082039 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.082004    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"]
Jul 07 02:24:01.084626 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.084599    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-95xxp"]
Jul 07 02:24:01.084812 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.084711    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.089788 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.089766    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 07 02:24:01.090609 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.090591    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-vds8d\""
Jul 07 02:24:01.100246 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.100221    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j54wz\" (UniqueName: \"kubernetes.io/projected/05832995-62fc-4e07-8c79-2ca962e7702e-kube-api-access-j54wz\") pod \"kserve-controller-manager-f8f7445f4-95xxp\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") " pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.100348 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.100268    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05832995-62fc-4e07-8c79-2ca962e7702e-cert\") pod \"kserve-controller-manager-f8f7445f4-95xxp\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") " pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.117245 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.117209    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"]
Jul 07 02:24:01.200624 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.200586    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j54wz\" (UniqueName: \"kubernetes.io/projected/05832995-62fc-4e07-8c79-2ca962e7702e-kube-api-access-j54wz\") pod \"kserve-controller-manager-f8f7445f4-95xxp\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") " pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.200793 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.200632    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c398fcf6-4565-46f8-9fcc-8575fa10cfb4-cert\") pod \"llmisvc-controller-manager-67d95fdc48-qtbw2\" (UID: \"c398fcf6-4565-46f8-9fcc-8575fa10cfb4\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.200793 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.200665    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwrsv\" (UniqueName: \"kubernetes.io/projected/c398fcf6-4565-46f8-9fcc-8575fa10cfb4-kube-api-access-cwrsv\") pod \"llmisvc-controller-manager-67d95fdc48-qtbw2\" (UID: \"c398fcf6-4565-46f8-9fcc-8575fa10cfb4\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.200793 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.200689    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05832995-62fc-4e07-8c79-2ca962e7702e-cert\") pod \"kserve-controller-manager-f8f7445f4-95xxp\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") " pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.203196 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.203166    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05832995-62fc-4e07-8c79-2ca962e7702e-cert\") pod \"kserve-controller-manager-f8f7445f4-95xxp\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") " pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.211771 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.211748    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j54wz\" (UniqueName: \"kubernetes.io/projected/05832995-62fc-4e07-8c79-2ca962e7702e-kube-api-access-j54wz\") pod \"kserve-controller-manager-f8f7445f4-95xxp\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") " pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.301071 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.301033    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c398fcf6-4565-46f8-9fcc-8575fa10cfb4-cert\") pod \"llmisvc-controller-manager-67d95fdc48-qtbw2\" (UID: \"c398fcf6-4565-46f8-9fcc-8575fa10cfb4\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.301241 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.301082    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwrsv\" (UniqueName: \"kubernetes.io/projected/c398fcf6-4565-46f8-9fcc-8575fa10cfb4-kube-api-access-cwrsv\") pod \"llmisvc-controller-manager-67d95fdc48-qtbw2\" (UID: \"c398fcf6-4565-46f8-9fcc-8575fa10cfb4\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.303699 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.303668    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c398fcf6-4565-46f8-9fcc-8575fa10cfb4-cert\") pod \"llmisvc-controller-manager-67d95fdc48-qtbw2\" (UID: \"c398fcf6-4565-46f8-9fcc-8575fa10cfb4\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.311075 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.311016    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwrsv\" (UniqueName: \"kubernetes.io/projected/c398fcf6-4565-46f8-9fcc-8575fa10cfb4-kube-api-access-cwrsv\") pod \"llmisvc-controller-manager-67d95fdc48-qtbw2\" (UID: \"c398fcf6-4565-46f8-9fcc-8575fa10cfb4\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.369727 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.369677    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:01.396334 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.396305    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:01.516697 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.516663    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-95xxp"]
Jul 07 02:24:01.520614 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:24:01.520580    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05832995_62fc_4e07_8c79_2ca962e7702e.slice/crio-f64fe2e4a52b9686fbb361f9711e2b54b7f37e5a83bf44afdaac71b53c2ec1f3 WatchSource:0}: Error finding container f64fe2e4a52b9686fbb361f9711e2b54b7f37e5a83bf44afdaac71b53c2ec1f3: Status 404 returned error can't find the container with id f64fe2e4a52b9686fbb361f9711e2b54b7f37e5a83bf44afdaac71b53c2ec1f3
Jul 07 02:24:01.522071 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.522051    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 02:24:01.544032 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.544000    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"]
Jul 07 02:24:01.546809 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:24:01.546784    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc398fcf6_4565_46f8_9fcc_8575fa10cfb4.slice/crio-70855db3db3042343138762057d79467c80f553a545355f8f89e2bfed91ac3ca WatchSource:0}: Error finding container 70855db3db3042343138762057d79467c80f553a545355f8f89e2bfed91ac3ca: Status 404 returned error can't find the container with id 70855db3db3042343138762057d79467c80f553a545355f8f89e2bfed91ac3ca
Jul 07 02:24:01.973858 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.973811    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp" event={"ID":"05832995-62fc-4e07-8c79-2ca962e7702e","Type":"ContainerStarted","Data":"f64fe2e4a52b9686fbb361f9711e2b54b7f37e5a83bf44afdaac71b53c2ec1f3"}
Jul 07 02:24:01.975205 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:01.975175    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2" event={"ID":"c398fcf6-4565-46f8-9fcc-8575fa10cfb4","Type":"ContainerStarted","Data":"70855db3db3042343138762057d79467c80f553a545355f8f89e2bfed91ac3ca"}
Jul 07 02:24:04.990020 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:04.989987    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp" event={"ID":"05832995-62fc-4e07-8c79-2ca962e7702e","Type":"ContainerStarted","Data":"dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc"}
Jul 07 02:24:04.990435 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:04.990084    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:04.991536 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:04.991515    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2" event={"ID":"c398fcf6-4565-46f8-9fcc-8575fa10cfb4","Type":"ContainerStarted","Data":"31058ba923bfeece67fc0d7c8dfa39eebd42ec3882d159942d354ce42e269560"}
Jul 07 02:24:04.991667 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:04.991651    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:05.007235 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:05.007189    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp" podStartSLOduration=0.932426147 podStartE2EDuration="4.007166799s" totalImagesPullingTime="3.074740652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:24:05.006963771 +0000 UTC m=+371.135050473" watchObservedRunningTime="2026-07-07 02:24:05.007166799 +0000 UTC m=+371.135253524"
Jul 07 02:24:05.027916 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:05.027855    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2" podStartSLOduration=0.979612408 podStartE2EDuration="4.027831285s" totalImagesPullingTime="3.048218877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:24:05.024951077 +0000 UTC m=+371.153037768" watchObservedRunningTime="2026-07-07 02:24:05.027831285 +0000 UTC m=+371.155917977"
Jul 07 02:24:35.997834 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:35.997803    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-qtbw2"
Jul 07 02:24:36.000855 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:36.000829    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:37.475412 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.475378    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-95xxp"]
Jul 07 02:24:37.475858 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.475576    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp" podUID="05832995-62fc-4e07-8c79-2ca962e7702e" containerName="manager" containerID="cri-o://dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc" gracePeriod=10
Jul 07 02:24:37.496999 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.496974    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-h4545"]
Jul 07 02:24:37.499168 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.499152    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.510645 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.510622    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-h4545"]
Jul 07 02:24:37.597911 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.597885    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dcf4a39-67aa-4538-a7ca-5ed5824ca25d-cert\") pod \"kserve-controller-manager-f8f7445f4-h4545\" (UID: \"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d\") " pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.598033 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.597933    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78ggh\" (UniqueName: \"kubernetes.io/projected/5dcf4a39-67aa-4538-a7ca-5ed5824ca25d-kube-api-access-78ggh\") pod \"kserve-controller-manager-f8f7445f4-h4545\" (UID: \"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d\") " pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.698791 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.698753    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dcf4a39-67aa-4538-a7ca-5ed5824ca25d-cert\") pod \"kserve-controller-manager-f8f7445f4-h4545\" (UID: \"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d\") " pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.698951 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.698827    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78ggh\" (UniqueName: \"kubernetes.io/projected/5dcf4a39-67aa-4538-a7ca-5ed5824ca25d-kube-api-access-78ggh\") pod \"kserve-controller-manager-f8f7445f4-h4545\" (UID: \"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d\") " pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.701480 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.701436    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dcf4a39-67aa-4538-a7ca-5ed5824ca25d-cert\") pod \"kserve-controller-manager-f8f7445f4-h4545\" (UID: \"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d\") " pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.707694 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.707669    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78ggh\" (UniqueName: \"kubernetes.io/projected/5dcf4a39-67aa-4538-a7ca-5ed5824ca25d-kube-api-access-78ggh\") pod \"kserve-controller-manager-f8f7445f4-h4545\" (UID: \"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d\") " pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.724702 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.724676    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:37.854830 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.854724    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:37.900886 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.900796    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j54wz\" (UniqueName: \"kubernetes.io/projected/05832995-62fc-4e07-8c79-2ca962e7702e-kube-api-access-j54wz\") pod \"05832995-62fc-4e07-8c79-2ca962e7702e\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") "
Jul 07 02:24:37.901032 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.900949    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05832995-62fc-4e07-8c79-2ca962e7702e-cert\") pod \"05832995-62fc-4e07-8c79-2ca962e7702e\" (UID: \"05832995-62fc-4e07-8c79-2ca962e7702e\") "
Jul 07 02:24:37.903773 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.903747    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05832995-62fc-4e07-8c79-2ca962e7702e-cert" (OuterVolumeSpecName: "cert") pod "05832995-62fc-4e07-8c79-2ca962e7702e" (UID: "05832995-62fc-4e07-8c79-2ca962e7702e"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:24:37.903903 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.903836    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05832995-62fc-4e07-8c79-2ca962e7702e-kube-api-access-j54wz" (OuterVolumeSpecName: "kube-api-access-j54wz") pod "05832995-62fc-4e07-8c79-2ca962e7702e" (UID: "05832995-62fc-4e07-8c79-2ca962e7702e"). InnerVolumeSpecName "kube-api-access-j54wz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:24:37.989742 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:37.989715    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-h4545"]
Jul 07 02:24:37.992410 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:24:37.992384    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcf4a39_67aa_4538_a7ca_5ed5824ca25d.slice/crio-51af926f32e2da141cefa65fbfa20de155ab032ffb234c63c93703266f8ef1e7 WatchSource:0}: Error finding container 51af926f32e2da141cefa65fbfa20de155ab032ffb234c63c93703266f8ef1e7: Status 404 returned error can't find the container with id 51af926f32e2da141cefa65fbfa20de155ab032ffb234c63c93703266f8ef1e7
Jul 07 02:24:38.001626 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.001604    2585 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05832995-62fc-4e07-8c79-2ca962e7702e-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:24:38.001715 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.001627    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j54wz\" (UniqueName: \"kubernetes.io/projected/05832995-62fc-4e07-8c79-2ca962e7702e-kube-api-access-j54wz\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:24:38.128018 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.127924    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-f8f7445f4-h4545" event={"ID":"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d","Type":"ContainerStarted","Data":"51af926f32e2da141cefa65fbfa20de155ab032ffb234c63c93703266f8ef1e7"}
Jul 07 02:24:38.129106 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.129075    2585 generic.go:356] "Generic (PLEG): container finished" podID="05832995-62fc-4e07-8c79-2ca962e7702e" containerID="dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc" exitCode=0
Jul 07 02:24:38.129247 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.129122    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp"
Jul 07 02:24:38.129247 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.129131    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp" event={"ID":"05832995-62fc-4e07-8c79-2ca962e7702e","Type":"ContainerDied","Data":"dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc"}
Jul 07 02:24:38.129247 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.129176    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-f8f7445f4-95xxp" event={"ID":"05832995-62fc-4e07-8c79-2ca962e7702e","Type":"ContainerDied","Data":"f64fe2e4a52b9686fbb361f9711e2b54b7f37e5a83bf44afdaac71b53c2ec1f3"}
Jul 07 02:24:38.129247 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.129207    2585 scope.go:117] "RemoveContainer" containerID="dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc"
Jul 07 02:24:38.137835 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.137815    2585 scope.go:117] "RemoveContainer" containerID="dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc"
Jul 07 02:24:38.138112 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:24:38.138092    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc\": container with ID starting with dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc not found: ID does not exist" containerID="dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc"
Jul 07 02:24:38.138172 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.138125    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc"} err="failed to get container status \"dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc\": rpc error: code = NotFound desc = could not find container \"dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc\": container with ID starting with dccac63427b006cf66ab99cae5277f09804735c1c745bd94fcb44071370789bc not found: ID does not exist"
Jul 07 02:24:38.151686 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.151660    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-95xxp"]
Jul 07 02:24:38.157359 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.157339    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-f8f7445f4-95xxp"]
Jul 07 02:24:38.402125 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:38.402031    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05832995-62fc-4e07-8c79-2ca962e7702e" path="/var/lib/kubelet/pods/05832995-62fc-4e07-8c79-2ca962e7702e/volumes"
Jul 07 02:24:39.135106 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:39.135067    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-f8f7445f4-h4545" event={"ID":"5dcf4a39-67aa-4538-a7ca-5ed5824ca25d","Type":"ContainerStarted","Data":"a7f63ccd05bd8a3172b7a7f0c550af5724b6790e1bbb42a4821a756f6900bf06"}
Jul 07 02:24:39.135575 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:39.135173    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:24:39.152208 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:39.152152    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-f8f7445f4-h4545" podStartSLOduration=1.8409440990000001 podStartE2EDuration="2.152138467s" totalImagesPullingTime="311.194368ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:24:39.151550159 +0000 UTC m=+405.279636849" watchObservedRunningTime="2026-07-07 02:24:39.152138467 +0000 UTC m=+405.280225157"
Jul 07 02:24:43.288804 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:24:43.288767    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d54766466-mwsl8"]
Jul 07 02:25:08.316740 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.316675    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7d54766466-mwsl8" podUID="79cdf817-f691-458b-83c6-b628d6e1295d" containerName="console" containerID="cri-o://78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1" gracePeriod=15
Jul 07 02:25:08.563463 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.563430    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:25:08.651591 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651500    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-oauth-config\") pod \"79cdf817-f691-458b-83c6-b628d6e1295d\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") "
Jul 07 02:25:08.651591 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651536    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-oauth-serving-cert\") pod \"79cdf817-f691-458b-83c6-b628d6e1295d\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") "
Jul 07 02:25:08.651591 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651557    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-service-ca\") pod \"79cdf817-f691-458b-83c6-b628d6e1295d\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") "
Jul 07 02:25:08.651840 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651594    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pghqr\" (UniqueName: \"kubernetes.io/projected/79cdf817-f691-458b-83c6-b628d6e1295d-kube-api-access-pghqr\") pod \"79cdf817-f691-458b-83c6-b628d6e1295d\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") "
Jul 07 02:25:08.651840 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651644    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-console-config\") pod \"79cdf817-f691-458b-83c6-b628d6e1295d\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") "
Jul 07 02:25:08.651840 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651683    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-serving-cert\") pod \"79cdf817-f691-458b-83c6-b628d6e1295d\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") "
Jul 07 02:25:08.651840 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651710    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-trusted-ca-bundle\") pod \"79cdf817-f691-458b-83c6-b628d6e1295d\" (UID: \"79cdf817-f691-458b-83c6-b628d6e1295d\") "
Jul 07 02:25:08.652040 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651942    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "79cdf817-f691-458b-83c6-b628d6e1295d" (UID: "79cdf817-f691-458b-83c6-b628d6e1295d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:25:08.652040 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.651975    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-service-ca" (OuterVolumeSpecName: "service-ca") pod "79cdf817-f691-458b-83c6-b628d6e1295d" (UID: "79cdf817-f691-458b-83c6-b628d6e1295d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:25:08.652182 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.652151    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-console-config" (OuterVolumeSpecName: "console-config") pod "79cdf817-f691-458b-83c6-b628d6e1295d" (UID: "79cdf817-f691-458b-83c6-b628d6e1295d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:25:08.652307 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.652205    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "79cdf817-f691-458b-83c6-b628d6e1295d" (UID: "79cdf817-f691-458b-83c6-b628d6e1295d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:25:08.653879 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.653859    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79cdf817-f691-458b-83c6-b628d6e1295d-kube-api-access-pghqr" (OuterVolumeSpecName: "kube-api-access-pghqr") pod "79cdf817-f691-458b-83c6-b628d6e1295d" (UID: "79cdf817-f691-458b-83c6-b628d6e1295d"). InnerVolumeSpecName "kube-api-access-pghqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:25:08.654367 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.654337    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "79cdf817-f691-458b-83c6-b628d6e1295d" (UID: "79cdf817-f691-458b-83c6-b628d6e1295d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:25:08.654439 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.654371    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "79cdf817-f691-458b-83c6-b628d6e1295d" (UID: "79cdf817-f691-458b-83c6-b628d6e1295d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:25:08.753270 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.753228    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-serving-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:25:08.753270 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.753262    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-trusted-ca-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:25:08.753270 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.753275    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdf817-f691-458b-83c6-b628d6e1295d-console-oauth-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:25:08.753540 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.753286    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-oauth-serving-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:25:08.753540 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.753299    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-service-ca\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:25:08.753540 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.753311    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pghqr\" (UniqueName: \"kubernetes.io/projected/79cdf817-f691-458b-83c6-b628d6e1295d-kube-api-access-pghqr\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:25:08.753540 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:08.753323    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdf817-f691-458b-83c6-b628d6e1295d-console-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:25:09.256943 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.256904    2585 generic.go:356] "Generic (PLEG): container finished" podID="79cdf817-f691-458b-83c6-b628d6e1295d" containerID="78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1" exitCode=0
Jul 07 02:25:09.257137 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.256953    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d54766466-mwsl8"
Jul 07 02:25:09.257137 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.256969    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d54766466-mwsl8" event={"ID":"79cdf817-f691-458b-83c6-b628d6e1295d","Type":"ContainerDied","Data":"78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1"}
Jul 07 02:25:09.257137 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.257018    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d54766466-mwsl8" event={"ID":"79cdf817-f691-458b-83c6-b628d6e1295d","Type":"ContainerDied","Data":"957937c43eb8dafa276cc3c84a7f4e263814a31da4a2fbdea9a3608858e47c80"}
Jul 07 02:25:09.257137 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.257044    2585 scope.go:117] "RemoveContainer" containerID="78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1"
Jul 07 02:25:09.265436 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.265419    2585 scope.go:117] "RemoveContainer" containerID="78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1"
Jul 07 02:25:09.265727 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:25:09.265709    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1\": container with ID starting with 78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1 not found: ID does not exist" containerID="78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1"
Jul 07 02:25:09.265774 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.265737    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1"} err="failed to get container status \"78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1\": rpc error: code = NotFound desc = could not find container \"78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1\": container with ID starting with 78279f32caeb851f02370ede97235548def38ec5aa8b9b273e5e8ad12064e5c1 not found: ID does not exist"
Jul 07 02:25:09.277735 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.277708    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d54766466-mwsl8"]
Jul 07 02:25:09.281569 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:09.281551    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d54766466-mwsl8"]
Jul 07 02:25:10.145554 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:10.145523    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-f8f7445f4-h4545"
Jul 07 02:25:10.397187 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:25:10.397107    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79cdf817-f691-458b-83c6-b628d6e1295d" path="/var/lib/kubelet/pods/79cdf817-f691-458b-83c6-b628d6e1295d/volumes"
Jul 07 02:26:58.558271 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.558185    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"]
Jul 07 02:26:58.558848 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.558630    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79cdf817-f691-458b-83c6-b628d6e1295d" containerName="console"
Jul 07 02:26:58.558848 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.558651    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="79cdf817-f691-458b-83c6-b628d6e1295d" containerName="console"
Jul 07 02:26:58.558848 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.558682    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05832995-62fc-4e07-8c79-2ca962e7702e" containerName="manager"
Jul 07 02:26:58.558848 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.558691    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="05832995-62fc-4e07-8c79-2ca962e7702e" containerName="manager"
Jul 07 02:26:58.558848 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.558780    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="05832995-62fc-4e07-8c79-2ca962e7702e" containerName="manager"
Jul 07 02:26:58.558848 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.558791    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="79cdf817-f691-458b-83c6-b628d6e1295d" containerName="console"
Jul 07 02:26:58.561927 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.561908    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.564850 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.564816    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2346e-predictor-serving-cert\""
Jul 07 02:26:58.564850 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.564816    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 02:26:58.565008 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.564876    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 07 02:26:58.565136 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.565117    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2346e-kube-rbac-proxy-sar-config\""
Jul 07 02:26:58.566360 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.566338    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gqpvr\""
Jul 07 02:26:58.571212 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.571190    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"]
Jul 07 02:26:58.690186 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.690157    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-proxy-tls\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.690186 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.690188    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.690470 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.690212    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvmx6\" (UniqueName: \"kubernetes.io/projected/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-api-access-bvmx6\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.790950 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.790907    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-proxy-tls\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.791151 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.790959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.791151 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.790996    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvmx6\" (UniqueName: \"kubernetes.io/projected/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-api-access-bvmx6\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.792019 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.791990    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.794075 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.794046    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-proxy-tls\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.801147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.801120    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvmx6\" (UniqueName: \"kubernetes.io/projected/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-api-access-bvmx6\") pod \"error-404-isvc-2346e-predictor-7bc756bd48-sbb2x\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") " pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:58.872812 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:58.872725    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:26:59.021750 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:59.021720    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"]
Jul 07 02:26:59.025403 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:26:59.025363    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda19c8dc_a81d_4173_98ba_92d4e1cc1dea.slice/crio-ce8847679fef88be30ebf5a6dcb6236d10a492ab77f50954c0eba17fabf8a357 WatchSource:0}: Error finding container ce8847679fef88be30ebf5a6dcb6236d10a492ab77f50954c0eba17fabf8a357: Status 404 returned error can't find the container with id ce8847679fef88be30ebf5a6dcb6236d10a492ab77f50954c0eba17fabf8a357
Jul 07 02:26:59.700662 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:26:59.700618    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" event={"ID":"da19c8dc-a81d-4173-98ba-92d4e1cc1dea","Type":"ContainerStarted","Data":"ce8847679fef88be30ebf5a6dcb6236d10a492ab77f50954c0eba17fabf8a357"}
Jul 07 02:27:11.764923 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:11.764839    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" event={"ID":"da19c8dc-a81d-4173-98ba-92d4e1cc1dea","Type":"ContainerStarted","Data":"e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1"}
Jul 07 02:27:14.779260 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:14.779223    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" event={"ID":"da19c8dc-a81d-4173-98ba-92d4e1cc1dea","Type":"ContainerStarted","Data":"9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb"}
Jul 07 02:27:14.779785 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:14.779364    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:27:15.783044 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:15.783009    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:27:15.784483 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:15.784435    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 02:27:16.787500 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:16.787464    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 02:27:21.792334 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:21.792300    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:27:21.792836 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:21.792805    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 02:27:21.812678 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:21.812631    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podStartSLOduration=9.009531444 podStartE2EDuration="23.812616248s" totalImagesPullingTime="14.803084804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:26:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:27:14.821034474 +0000 UTC m=+560.949121177" watchObservedRunningTime="2026-07-07 02:27:21.812616248 +0000 UTC m=+567.940702939"
Jul 07 02:27:31.793434 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:31.793387    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 02:27:41.793634 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:41.793589    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 02:27:51.793465 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:51.793416    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 02:27:54.306631 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:54.306601    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:27:54.309305 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:27:54.309282    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:28:01.793613 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:01.793582    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:28:38.913538 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.913460    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"]
Jul 07 02:28:38.914087 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.913692    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" containerID="cri-o://e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1" gracePeriod=30
Jul 07 02:28:38.914087 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.913764    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kube-rbac-proxy" containerID="cri-o://9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb" gracePeriod=30
Jul 07 02:28:38.929122 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.929097    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"]
Jul 07 02:28:38.931515 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.931499    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:38.936330 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.936311    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3ea7f-predictor-serving-cert\""
Jul 07 02:28:38.936747 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.936727    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3ea7f-kube-rbac-proxy-sar-config\""
Jul 07 02:28:38.949550 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.949528    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"]
Jul 07 02:28:38.997555 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.997509    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db1e761e-49c2-4e82-aae2-de9d61839988-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:38.997739 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.997565    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:38.997739 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:38.997626    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxfrl\" (UniqueName: \"kubernetes.io/projected/db1e761e-49c2-4e82-aae2-de9d61839988-kube-api-access-rxfrl\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.098723 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.098683    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db1e761e-49c2-4e82-aae2-de9d61839988-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.098934 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.098748    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.098934 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.098781    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxfrl\" (UniqueName: \"kubernetes.io/projected/db1e761e-49c2-4e82-aae2-de9d61839988-kube-api-access-rxfrl\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.099060 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:28:39.098999    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-serving-cert: secret "error-404-isvc-3ea7f-predictor-serving-cert" not found
Jul 07 02:28:39.099121 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:28:39.099073    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls podName:db1e761e-49c2-4e82-aae2-de9d61839988 nodeName:}" failed. No retries permitted until 2026-07-07 02:28:39.599052123 +0000 UTC m=+645.727138797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls") pod "error-404-isvc-3ea7f-predictor-8b8785f68-xf652" (UID: "db1e761e-49c2-4e82-aae2-de9d61839988") : secret "error-404-isvc-3ea7f-predictor-serving-cert" not found
Jul 07 02:28:39.099419 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.099401    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db1e761e-49c2-4e82-aae2-de9d61839988-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.108386 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.108366    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxfrl\" (UniqueName: \"kubernetes.io/projected/db1e761e-49c2-4e82-aae2-de9d61839988-kube-api-access-rxfrl\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.117952 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.117919    2585 generic.go:356] "Generic (PLEG): container finished" podID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerID="9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb" exitCode=2
Jul 07 02:28:39.118085 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.117994    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" event={"ID":"da19c8dc-a81d-4173-98ba-92d4e1cc1dea","Type":"ContainerDied","Data":"9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb"}
Jul 07 02:28:39.602350 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.602295    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.604811 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.604782    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls\") pod \"error-404-isvc-3ea7f-predictor-8b8785f68-xf652\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") " pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.841382 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.841336    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:39.981910 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:39.981882    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"]
Jul 07 02:28:39.983798 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:28:39.983764    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1e761e_49c2_4e82_aae2_de9d61839988.slice/crio-e14928657de93275bffb4a439880b085b71a0fe117fcbb7ef97afddee84f9cfe WatchSource:0}: Error finding container e14928657de93275bffb4a439880b085b71a0fe117fcbb7ef97afddee84f9cfe: Status 404 returned error can't find the container with id e14928657de93275bffb4a439880b085b71a0fe117fcbb7ef97afddee84f9cfe
Jul 07 02:28:40.124840 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:40.124800    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" event={"ID":"db1e761e-49c2-4e82-aae2-de9d61839988","Type":"ContainerStarted","Data":"21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f"}
Jul 07 02:28:40.125034 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:40.124873    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" event={"ID":"db1e761e-49c2-4e82-aae2-de9d61839988","Type":"ContainerStarted","Data":"f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262"}
Jul 07 02:28:40.125034 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:40.124912    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:40.125034 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:40.124937    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" event={"ID":"db1e761e-49c2-4e82-aae2-de9d61839988","Type":"ContainerStarted","Data":"e14928657de93275bffb4a439880b085b71a0fe117fcbb7ef97afddee84f9cfe"}
Jul 07 02:28:40.146336 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:40.146225    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podStartSLOduration=2.146209077 podStartE2EDuration="2.146209077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:28:40.144538276 +0000 UTC m=+646.272624969" watchObservedRunningTime="2026-07-07 02:28:40.146209077 +0000 UTC m=+646.274295767"
Jul 07 02:28:41.130049 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:41.130018    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:41.131327 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:41.131291    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 07 02:28:41.788552 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:41.788503    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 07 02:28:41.792825 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:41.792796    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 02:28:42.134488 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.134438    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 07 02:28:42.365643 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.365618    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:28:42.524533 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.524489    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvmx6\" (UniqueName: \"kubernetes.io/projected/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-api-access-bvmx6\") pod \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") "
Jul 07 02:28:42.524533 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.524539    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-rbac-proxy-sar-config\") pod \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") "
Jul 07 02:28:42.524797 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.524573    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-proxy-tls\") pod \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\" (UID: \"da19c8dc-a81d-4173-98ba-92d4e1cc1dea\") "
Jul 07 02:28:42.524917 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.524893    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "da19c8dc-a81d-4173-98ba-92d4e1cc1dea" (UID: "da19c8dc-a81d-4173-98ba-92d4e1cc1dea"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:28:42.526945 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.526916    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "da19c8dc-a81d-4173-98ba-92d4e1cc1dea" (UID: "da19c8dc-a81d-4173-98ba-92d4e1cc1dea"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:28:42.527073 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.526948    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-api-access-bvmx6" (OuterVolumeSpecName: "kube-api-access-bvmx6") pod "da19c8dc-a81d-4173-98ba-92d4e1cc1dea" (UID: "da19c8dc-a81d-4173-98ba-92d4e1cc1dea"). InnerVolumeSpecName "kube-api-access-bvmx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:28:42.625730 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.625693    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:28:42.625730 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.625720    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvmx6\" (UniqueName: \"kubernetes.io/projected/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-api-access-bvmx6\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:28:42.625730 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:42.625730    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/da19c8dc-a81d-4173-98ba-92d4e1cc1dea-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:28:43.140603 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.140571    2585 generic.go:356] "Generic (PLEG): container finished" podID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerID="e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1" exitCode=0
Jul 07 02:28:43.141024 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.140644    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"
Jul 07 02:28:43.141024 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.140647    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" event={"ID":"da19c8dc-a81d-4173-98ba-92d4e1cc1dea","Type":"ContainerDied","Data":"e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1"}
Jul 07 02:28:43.141024 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.140792    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x" event={"ID":"da19c8dc-a81d-4173-98ba-92d4e1cc1dea","Type":"ContainerDied","Data":"ce8847679fef88be30ebf5a6dcb6236d10a492ab77f50954c0eba17fabf8a357"}
Jul 07 02:28:43.141024 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.140818    2585 scope.go:117] "RemoveContainer" containerID="9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb"
Jul 07 02:28:43.149633 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.149611    2585 scope.go:117] "RemoveContainer" containerID="e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1"
Jul 07 02:28:43.157478 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.157455    2585 scope.go:117] "RemoveContainer" containerID="9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb"
Jul 07 02:28:43.157755 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:28:43.157734    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb\": container with ID starting with 9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb not found: ID does not exist" containerID="9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb"
Jul 07 02:28:43.157802 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.157763    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb"} err="failed to get container status \"9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb\": rpc error: code = NotFound desc = could not find container \"9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb\": container with ID starting with 9b9a0245a9358660fe339d029104a40852a3cb7d2bb69aa9359c7600080bccbb not found: ID does not exist"
Jul 07 02:28:43.157802 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.157779    2585 scope.go:117] "RemoveContainer" containerID="e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1"
Jul 07 02:28:43.158059 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:28:43.158041    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1\": container with ID starting with e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1 not found: ID does not exist" containerID="e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1"
Jul 07 02:28:43.158100 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.158065    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1"} err="failed to get container status \"e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1\": rpc error: code = NotFound desc = could not find container \"e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1\": container with ID starting with e36ae783a70f23e36d358d5a44f9c4e7bbd68ccde84396b89177bfa2d18e2ad1 not found: ID does not exist"
Jul 07 02:28:43.164438 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.164415    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"]
Jul 07 02:28:43.169035 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:43.169014    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2346e-predictor-7bc756bd48-sbb2x"]
Jul 07 02:28:44.396730 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:44.396698    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" path="/var/lib/kubelet/pods/da19c8dc-a81d-4173-98ba-92d4e1cc1dea/volumes"
Jul 07 02:28:47.138597 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:47.138569    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:28:47.138971 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:47.138947    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 07 02:28:57.139382 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:28:57.139339    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 07 02:29:07.139434 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:07.139391    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 07 02:29:17.139498 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:17.139428    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 07 02:29:18.704339 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.704300    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"]
Jul 07 02:29:18.704834 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.704748    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kube-rbac-proxy"
Jul 07 02:29:18.704834 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.704767    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kube-rbac-proxy"
Jul 07 02:29:18.704834 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.704781    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container"
Jul 07 02:29:18.704834 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.704789    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container"
Jul 07 02:29:18.705040 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.704862    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kserve-container"
Jul 07 02:29:18.705040 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.704878    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="da19c8dc-a81d-4173-98ba-92d4e1cc1dea" containerName="kube-rbac-proxy"
Jul 07 02:29:18.706812 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.706791    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.709789 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.709766    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-821ea-kube-rbac-proxy-sar-config\""
Jul 07 02:29:18.710540 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.710518    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-821ea-predictor-serving-cert\""
Jul 07 02:29:18.719849 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.719827    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"]
Jul 07 02:29:18.788495 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.788441    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n98vl\" (UniqueName: \"kubernetes.io/projected/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-api-access-n98vl\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.788668 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.788513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-proxy-tls\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.788668 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.788548    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.889037 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.888998    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-proxy-tls\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.889228 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.889043    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.889228 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.889086    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n98vl\" (UniqueName: \"kubernetes.io/projected/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-api-access-n98vl\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.889901 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.889821    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.891655 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.891626    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-proxy-tls\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:18.899140 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:18.899111    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n98vl\" (UniqueName: \"kubernetes.io/projected/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-api-access-n98vl\") pod \"error-404-isvc-821ea-predictor-68c7476dc4-gs9p5\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") " pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:19.017386 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.017289    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:19.158124 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.155092    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"]
Jul 07 02:29:19.294968 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.294880    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" event={"ID":"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d","Type":"ContainerStarted","Data":"2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e"}
Jul 07 02:29:19.294968 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.294954    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" event={"ID":"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d","Type":"ContainerStarted","Data":"051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb"}
Jul 07 02:29:19.295147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.294981    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:19.295147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.295004    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" event={"ID":"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d","Type":"ContainerStarted","Data":"37fe84a6e484dd2abcbeee10f9eb417df0fc510df588b596a210458829f6c797"}
Jul 07 02:29:19.295147 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.295031    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:19.296421 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.296393    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 07 02:29:19.315389 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:19.315336    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podStartSLOduration=1.315319645 podStartE2EDuration="1.315319645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:29:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:29:19.3131717 +0000 UTC m=+685.441258391" watchObservedRunningTime="2026-07-07 02:29:19.315319645 +0000 UTC m=+685.443406336"
Jul 07 02:29:20.300241 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:20.300203    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 07 02:29:25.305270 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:25.305231    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:29:25.305905 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:25.305879    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 07 02:29:27.140346 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:27.140317    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:29:35.306306 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:35.306263    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 07 02:29:45.306806 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:45.306718    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 07 02:29:55.306601 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:29:55.306562    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 07 02:30:05.307470 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:30:05.307424    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:32:54.329271 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:32:54.329240    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:32:54.332563 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:32:54.332545    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:37:54.352040 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:37:54.352009    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:37:54.356069 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:37:54.356050    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:38:03.575220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.575181    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"]
Jul 07 02:38:03.575705 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.575479    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container" containerID="cri-o://f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262" gracePeriod=30
Jul 07 02:38:03.575705 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.575550    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kube-rbac-proxy" containerID="cri-o://21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f" gracePeriod=30
Jul 07 02:38:03.689554 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.689525    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"]
Jul 07 02:38:03.692200 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.692184    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.695134 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.695095    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-cec47-predictor-serving-cert\""
Jul 07 02:38:03.695272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.695148    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-cec47-kube-rbac-proxy-sar-config\""
Jul 07 02:38:03.703536 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.703464    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"]
Jul 07 02:38:03.772364 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.772325    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.772364 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.772364    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6310814-479f-42c5-9eba-995181462a21-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.772641 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.772386    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5758\" (UniqueName: \"kubernetes.io/projected/a6310814-479f-42c5-9eba-995181462a21-kube-api-access-m5758\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.873623 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.873543    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.873623 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.873576    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6310814-479f-42c5-9eba-995181462a21-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.873623 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.873599    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5758\" (UniqueName: \"kubernetes.io/projected/a6310814-479f-42c5-9eba-995181462a21-kube-api-access-m5758\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.873869 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:03.873705    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-cec47-predictor-serving-cert: secret "error-404-isvc-cec47-predictor-serving-cert" not found
Jul 07 02:38:03.873869 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:03.873778    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls podName:a6310814-479f-42c5-9eba-995181462a21 nodeName:}" failed. No retries permitted until 2026-07-07 02:38:04.373756648 +0000 UTC m=+1210.501843319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls") pod "error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" (UID: "a6310814-479f-42c5-9eba-995181462a21") : secret "error-404-isvc-cec47-predictor-serving-cert" not found
Jul 07 02:38:03.874381 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.874352    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6310814-479f-42c5-9eba-995181462a21-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:03.885708 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:03.885683    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5758\" (UniqueName: \"kubernetes.io/projected/a6310814-479f-42c5-9eba-995181462a21-kube-api-access-m5758\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:04.379038 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:04.378998    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:04.381597 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:04.381565    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls\") pod \"error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") " pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:04.478535 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:04.478501    2585 generic.go:356] "Generic (PLEG): container finished" podID="db1e761e-49c2-4e82-aae2-de9d61839988" containerID="21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f" exitCode=2
Jul 07 02:38:04.478699 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:04.478572    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" event={"ID":"db1e761e-49c2-4e82-aae2-de9d61839988","Type":"ContainerDied","Data":"21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f"}
Jul 07 02:38:04.607783 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:04.607758    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:04.741650 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:04.741627    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"]
Jul 07 02:38:04.743827 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:38:04.743795    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6310814_479f_42c5_9eba_995181462a21.slice/crio-38cbcfc5128c16c11fdcdfdf694eba95843563f4e13258c22c13b7e7b08bf078 WatchSource:0}: Error finding container 38cbcfc5128c16c11fdcdfdf694eba95843563f4e13258c22c13b7e7b08bf078: Status 404 returned error can't find the container with id 38cbcfc5128c16c11fdcdfdf694eba95843563f4e13258c22c13b7e7b08bf078
Jul 07 02:38:05.484378 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:05.484340    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" event={"ID":"a6310814-479f-42c5-9eba-995181462a21","Type":"ContainerStarted","Data":"b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968"}
Jul 07 02:38:05.484559 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:05.484394    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" event={"ID":"a6310814-479f-42c5-9eba-995181462a21","Type":"ContainerStarted","Data":"5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474"}
Jul 07 02:38:05.484559 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:05.484414    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" event={"ID":"a6310814-479f-42c5-9eba-995181462a21","Type":"ContainerStarted","Data":"38cbcfc5128c16c11fdcdfdf694eba95843563f4e13258c22c13b7e7b08bf078"}
Jul 07 02:38:05.484559 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:05.484494    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:05.504515 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:05.504437    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podStartSLOduration=2.50442112 podStartE2EDuration="2.50442112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:38:05.502693485 +0000 UTC m=+1211.630780176" watchObservedRunningTime="2026-07-07 02:38:05.50442112 +0000 UTC m=+1211.632507875"
Jul 07 02:38:06.489064 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.489032    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:06.490471 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.490427    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 07 02:38:06.832982 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.832957    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:38:06.898341 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.898312    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls\") pod \"db1e761e-49c2-4e82-aae2-de9d61839988\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") "
Jul 07 02:38:06.898550 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.898353    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db1e761e-49c2-4e82-aae2-de9d61839988-kube-rbac-proxy-sar-config\") pod \"db1e761e-49c2-4e82-aae2-de9d61839988\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") "
Jul 07 02:38:06.898550 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.898403    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxfrl\" (UniqueName: \"kubernetes.io/projected/db1e761e-49c2-4e82-aae2-de9d61839988-kube-api-access-rxfrl\") pod \"db1e761e-49c2-4e82-aae2-de9d61839988\" (UID: \"db1e761e-49c2-4e82-aae2-de9d61839988\") "
Jul 07 02:38:06.898811 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.898782    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db1e761e-49c2-4e82-aae2-de9d61839988-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "db1e761e-49c2-4e82-aae2-de9d61839988" (UID: "db1e761e-49c2-4e82-aae2-de9d61839988"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:38:06.900621 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.900584    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "db1e761e-49c2-4e82-aae2-de9d61839988" (UID: "db1e761e-49c2-4e82-aae2-de9d61839988"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:38:06.900828 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.900807    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1e761e-49c2-4e82-aae2-de9d61839988-kube-api-access-rxfrl" (OuterVolumeSpecName: "kube-api-access-rxfrl") pod "db1e761e-49c2-4e82-aae2-de9d61839988" (UID: "db1e761e-49c2-4e82-aae2-de9d61839988"). InnerVolumeSpecName "kube-api-access-rxfrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:38:06.999914 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.999825    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxfrl\" (UniqueName: \"kubernetes.io/projected/db1e761e-49c2-4e82-aae2-de9d61839988-kube-api-access-rxfrl\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:38:06.999914 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.999858    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db1e761e-49c2-4e82-aae2-de9d61839988-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:38:06.999914 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:06.999867    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db1e761e-49c2-4e82-aae2-de9d61839988-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:38:07.494689 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.494654    2585 generic.go:356] "Generic (PLEG): container finished" podID="db1e761e-49c2-4e82-aae2-de9d61839988" containerID="f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262" exitCode=0
Jul 07 02:38:07.495165 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.494713    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"
Jul 07 02:38:07.495165 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.494739    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" event={"ID":"db1e761e-49c2-4e82-aae2-de9d61839988","Type":"ContainerDied","Data":"f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262"}
Jul 07 02:38:07.495165 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.494801    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652" event={"ID":"db1e761e-49c2-4e82-aae2-de9d61839988","Type":"ContainerDied","Data":"e14928657de93275bffb4a439880b085b71a0fe117fcbb7ef97afddee84f9cfe"}
Jul 07 02:38:07.495165 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.494828    2585 scope.go:117] "RemoveContainer" containerID="21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f"
Jul 07 02:38:07.495529 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.495499    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 07 02:38:07.503720 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.503702    2585 scope.go:117] "RemoveContainer" containerID="f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262"
Jul 07 02:38:07.510888 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.510869    2585 scope.go:117] "RemoveContainer" containerID="21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f"
Jul 07 02:38:07.511123 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:07.511104    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f\": container with ID starting with 21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f not found: ID does not exist" containerID="21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f"
Jul 07 02:38:07.511191 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.511138    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f"} err="failed to get container status \"21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f\": rpc error: code = NotFound desc = could not find container \"21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f\": container with ID starting with 21e062b42421af308735535b6c517eaec963d118b1683b043150ec1ff7d7d98f not found: ID does not exist"
Jul 07 02:38:07.511191 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.511161    2585 scope.go:117] "RemoveContainer" containerID="f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262"
Jul 07 02:38:07.511375 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:07.511360    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262\": container with ID starting with f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262 not found: ID does not exist" containerID="f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262"
Jul 07 02:38:07.511414 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.511383    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262"} err="failed to get container status \"f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262\": rpc error: code = NotFound desc = could not find container \"f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262\": container with ID starting with f5007ab1af81494f9b3cfddfcbafa9e9650445ae16609eb589a814340bd56262 not found: ID does not exist"
Jul 07 02:38:07.517618 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.517597    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"]
Jul 07 02:38:07.521498 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:07.521478    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ea7f-predictor-8b8785f68-xf652"]
Jul 07 02:38:08.396731 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:08.396694    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" path="/var/lib/kubelet/pods/db1e761e-49c2-4e82-aae2-de9d61839988/volumes"
Jul 07 02:38:12.500437 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:12.500408    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:38:12.500965 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:12.500938    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 07 02:38:22.501586 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:22.501541    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 07 02:38:32.501257 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:32.501219    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 07 02:38:33.589047 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.589005    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"]
Jul 07 02:38:33.589586 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.589378    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" containerID="cri-o://051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb" gracePeriod=30
Jul 07 02:38:33.589773 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.589716    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kube-rbac-proxy" containerID="cri-o://2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e" gracePeriod=30
Jul 07 02:38:33.712154 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.712119    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"]
Jul 07 02:38:33.712594 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.712579    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container"
Jul 07 02:38:33.712643 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.712598    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container"
Jul 07 02:38:33.712643 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.712610    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kube-rbac-proxy"
Jul 07 02:38:33.712643 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.712619    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kube-rbac-proxy"
Jul 07 02:38:33.712740 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.712703    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kserve-container"
Jul 07 02:38:33.712740 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.712713    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1e761e-49c2-4e82-aae2-de9d61839988" containerName="kube-rbac-proxy"
Jul 07 02:38:33.715921 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.715898    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.719092 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.719070    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-524cd-predictor-serving-cert\""
Jul 07 02:38:33.719208 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.719080    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-524cd-kube-rbac-proxy-sar-config\""
Jul 07 02:38:33.728523 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.728492    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"]
Jul 07 02:38:33.819715 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.819674    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.819715 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.819719    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md6bb\" (UniqueName: \"kubernetes.io/projected/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-api-access-md6bb\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.819932 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.819748    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.921014 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.920922    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.921014 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.920973    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md6bb\" (UniqueName: \"kubernetes.io/projected/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-api-access-md6bb\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.921014 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.921015    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.921303 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:33.921163    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-524cd-predictor-serving-cert: secret "error-404-isvc-524cd-predictor-serving-cert" not found
Jul 07 02:38:33.921303 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:33.921240    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls podName:86fa344a-3fb0-4d92-8c46-d3d4d24801a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:38:34.421216217 +0000 UTC m=+1240.549302903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls") pod "error-404-isvc-524cd-predictor-58fd54b564-2w825" (UID: "86fa344a-3fb0-4d92-8c46-d3d4d24801a5") : secret "error-404-isvc-524cd-predictor-serving-cert" not found
Jul 07 02:38:33.921814 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.921787    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:33.930563 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:33.930536    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md6bb\" (UniqueName: \"kubernetes.io/projected/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-api-access-md6bb\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:34.426387 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:34.426342    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:34.428904 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:34.428873    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls\") pod \"error-404-isvc-524cd-predictor-58fd54b564-2w825\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") " pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:34.611590 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:34.611543    2585 generic.go:356] "Generic (PLEG): container finished" podID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerID="2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e" exitCode=2
Jul 07 02:38:34.612024 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:34.611576    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" event={"ID":"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d","Type":"ContainerDied","Data":"2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e"}
Jul 07 02:38:34.630831 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:34.630798    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:34.764765 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:34.764737    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"]
Jul 07 02:38:34.767261 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:38:34.767238    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fa344a_3fb0_4d92_8c46_d3d4d24801a5.slice/crio-cc75620255fbee0a01baa17b35a48b6395f3d10433361ffe3ed78c01f2418cb2 WatchSource:0}: Error finding container cc75620255fbee0a01baa17b35a48b6395f3d10433361ffe3ed78c01f2418cb2: Status 404 returned error can't find the container with id cc75620255fbee0a01baa17b35a48b6395f3d10433361ffe3ed78c01f2418cb2
Jul 07 02:38:35.300683 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.300634    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Jul 07 02:38:35.305927 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.305904    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 07 02:38:35.617673 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.617577    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" event={"ID":"86fa344a-3fb0-4d92-8c46-d3d4d24801a5","Type":"ContainerStarted","Data":"f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed"}
Jul 07 02:38:35.617673 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.617635    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" event={"ID":"86fa344a-3fb0-4d92-8c46-d3d4d24801a5","Type":"ContainerStarted","Data":"dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d"}
Jul 07 02:38:35.618081 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.617755    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:35.618081 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.617808    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" event={"ID":"86fa344a-3fb0-4d92-8c46-d3d4d24801a5","Type":"ContainerStarted","Data":"cc75620255fbee0a01baa17b35a48b6395f3d10433361ffe3ed78c01f2418cb2"}
Jul 07 02:38:35.618081 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.617923    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:35.619432 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.619404    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 07 02:38:35.638699 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:35.638648    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podStartSLOduration=2.638630391 podStartE2EDuration="2.638630391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:38:35.637717451 +0000 UTC m=+1241.765804142" watchObservedRunningTime="2026-07-07 02:38:35.638630391 +0000 UTC m=+1241.766717104"
Jul 07 02:38:36.622211 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:36.622169    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 07 02:38:36.949458 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:36.949419    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:38:37.045987 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.045951    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n98vl\" (UniqueName: \"kubernetes.io/projected/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-api-access-n98vl\") pod \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") "
Jul 07 02:38:37.046183 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.046042    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-rbac-proxy-sar-config\") pod \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") "
Jul 07 02:38:37.046183 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.046069    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-proxy-tls\") pod \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\" (UID: \"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d\") "
Jul 07 02:38:37.046404 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.046380    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" (UID: "44e3421f-0f5c-4c2d-8993-c1d1556b7f1d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:38:37.048255 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.048238    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-api-access-n98vl" (OuterVolumeSpecName: "kube-api-access-n98vl") pod "44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" (UID: "44e3421f-0f5c-4c2d-8993-c1d1556b7f1d"). InnerVolumeSpecName "kube-api-access-n98vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:38:37.048315 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.048247    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" (UID: "44e3421f-0f5c-4c2d-8993-c1d1556b7f1d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:38:37.146766 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.146731    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n98vl\" (UniqueName: \"kubernetes.io/projected/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-api-access-n98vl\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:38:37.146766 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.146758    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:38:37.146766 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.146768    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:38:37.627524 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.627484    2585 generic.go:356] "Generic (PLEG): container finished" podID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerID="051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb" exitCode=0
Jul 07 02:38:37.628004 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.627553    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"
Jul 07 02:38:37.628004 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.627566    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" event={"ID":"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d","Type":"ContainerDied","Data":"051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb"}
Jul 07 02:38:37.628004 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.627631    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5" event={"ID":"44e3421f-0f5c-4c2d-8993-c1d1556b7f1d","Type":"ContainerDied","Data":"37fe84a6e484dd2abcbeee10f9eb417df0fc510df588b596a210458829f6c797"}
Jul 07 02:38:37.628004 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.627660    2585 scope.go:117] "RemoveContainer" containerID="2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e"
Jul 07 02:38:37.636473 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.636436    2585 scope.go:117] "RemoveContainer" containerID="051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb"
Jul 07 02:38:37.644043 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.644028    2585 scope.go:117] "RemoveContainer" containerID="2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e"
Jul 07 02:38:37.644274 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:37.644250    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e\": container with ID starting with 2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e not found: ID does not exist" containerID="2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e"
Jul 07 02:38:37.644311 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.644285    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e"} err="failed to get container status \"2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e\": rpc error: code = NotFound desc = could not find container \"2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e\": container with ID starting with 2edf029243c04747655467ab92424678d0a238111969dd7c931d5a322feaf02e not found: ID does not exist"
Jul 07 02:38:37.644311 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.644301    2585 scope.go:117] "RemoveContainer" containerID="051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb"
Jul 07 02:38:37.644569 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:38:37.644543    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb\": container with ID starting with 051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb not found: ID does not exist" containerID="051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb"
Jul 07 02:38:37.644612 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.644576    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb"} err="failed to get container status \"051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb\": rpc error: code = NotFound desc = could not find container \"051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb\": container with ID starting with 051a9d45ec99af390d86eb8ead23ea55ae0075c981fcfde36e8a70324c83bdfb not found: ID does not exist"
Jul 07 02:38:37.650808 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.650787    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"]
Jul 07 02:38:37.655091 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:37.655070    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-821ea-predictor-68c7476dc4-gs9p5"]
Jul 07 02:38:38.397898 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:38.397868    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" path="/var/lib/kubelet/pods/44e3421f-0f5c-4c2d-8993-c1d1556b7f1d/volumes"
Jul 07 02:38:41.627349 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:41.627321    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:38:41.627861 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:41.627765    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 07 02:38:42.501005 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:42.500971    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 07 02:38:51.628065 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:51.627979    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 07 02:38:52.502325 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:38:52.502286    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:39:01.628175 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:01.628134    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 07 02:39:11.628403 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:11.628365    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 07 02:39:13.829895 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:13.829863    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"]
Jul 07 02:39:13.830276 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:13.830096    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container" containerID="cri-o://5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474" gracePeriod=30
Jul 07 02:39:13.830276 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:13.830133    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kube-rbac-proxy" containerID="cri-o://b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968" gracePeriod=30
Jul 07 02:39:14.031132 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.031084    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"]
Jul 07 02:39:14.031585 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.031565    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container"
Jul 07 02:39:14.031585 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.031586    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container"
Jul 07 02:39:14.031719 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.031623    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kube-rbac-proxy"
Jul 07 02:39:14.031719 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.031632    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kube-rbac-proxy"
Jul 07 02:39:14.031719 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.031711    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kube-rbac-proxy"
Jul 07 02:39:14.031814 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.031722    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e3421f-0f5c-4c2d-8993-c1d1556b7f1d" containerName="kserve-container"
Jul 07 02:39:14.034996 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.034971    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.037920 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.037901    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-e4a8c-kube-rbac-proxy-sar-config\""
Jul 07 02:39:14.038033 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.037898    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-e4a8c-predictor-serving-cert\""
Jul 07 02:39:14.045046 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.045021    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"]
Jul 07 02:39:14.133395 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.133294    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.133395 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.133346    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00124b43-38ed-4308-a2f1-e997b3a23856-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.133597 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.133464    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngkx2\" (UniqueName: \"kubernetes.io/projected/00124b43-38ed-4308-a2f1-e997b3a23856-kube-api-access-ngkx2\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.234861 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.234816    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.234861 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.234863    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00124b43-38ed-4308-a2f1-e997b3a23856-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.235111 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.234921    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkx2\" (UniqueName: \"kubernetes.io/projected/00124b43-38ed-4308-a2f1-e997b3a23856-kube-api-access-ngkx2\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.235111 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:39:14.234975    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-serving-cert: secret "error-404-isvc-e4a8c-predictor-serving-cert" not found
Jul 07 02:39:14.235111 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:39:14.235044    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls podName:00124b43-38ed-4308-a2f1-e997b3a23856 nodeName:}" failed. No retries permitted until 2026-07-07 02:39:14.735028065 +0000 UTC m=+1280.863114733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls") pod "error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" (UID: "00124b43-38ed-4308-a2f1-e997b3a23856") : secret "error-404-isvc-e4a8c-predictor-serving-cert" not found
Jul 07 02:39:14.235556 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.235537    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00124b43-38ed-4308-a2f1-e997b3a23856-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.243908 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.243881    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngkx2\" (UniqueName: \"kubernetes.io/projected/00124b43-38ed-4308-a2f1-e997b3a23856-kube-api-access-ngkx2\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.739488 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.739427    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.741853 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.741828    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls\") pod \"error-404-isvc-e4a8c-predictor-669cdb968b-nfffg\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") " pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:14.781949 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.781914    2585 generic.go:356] "Generic (PLEG): container finished" podID="a6310814-479f-42c5-9eba-995181462a21" containerID="b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968" exitCode=2
Jul 07 02:39:14.782096 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.781973    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" event={"ID":"a6310814-479f-42c5-9eba-995181462a21","Type":"ContainerDied","Data":"b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968"}
Jul 07 02:39:14.946163 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:14.946121    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:15.078731 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:15.078653    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"]
Jul 07 02:39:15.081636 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:39:15.081602    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00124b43_38ed_4308_a2f1_e997b3a23856.slice/crio-f2f064adc1f0c44e935cf8d0eb36957e381337a17055fce48f87a8888686c41b WatchSource:0}: Error finding container f2f064adc1f0c44e935cf8d0eb36957e381337a17055fce48f87a8888686c41b: Status 404 returned error can't find the container with id f2f064adc1f0c44e935cf8d0eb36957e381337a17055fce48f87a8888686c41b
Jul 07 02:39:15.787673 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:15.787639    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" event={"ID":"00124b43-38ed-4308-a2f1-e997b3a23856","Type":"ContainerStarted","Data":"5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450"}
Jul 07 02:39:15.787855 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:15.787692    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" event={"ID":"00124b43-38ed-4308-a2f1-e997b3a23856","Type":"ContainerStarted","Data":"8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd"}
Jul 07 02:39:15.787855 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:15.787712    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" event={"ID":"00124b43-38ed-4308-a2f1-e997b3a23856","Type":"ContainerStarted","Data":"f2f064adc1f0c44e935cf8d0eb36957e381337a17055fce48f87a8888686c41b"}
Jul 07 02:39:15.787855 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:15.787782    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:15.807777 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:15.807731    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podStartSLOduration=1.807715197 podStartE2EDuration="1.807715197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:39:15.805473095 +0000 UTC m=+1281.933559784" watchObservedRunningTime="2026-07-07 02:39:15.807715197 +0000 UTC m=+1281.935801944"
Jul 07 02:39:16.791921 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:16.791886    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:16.793354 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:16.793326    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 07 02:39:17.084475 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.084432    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:39:17.159324 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.159291    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls\") pod \"a6310814-479f-42c5-9eba-995181462a21\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") "
Jul 07 02:39:17.159599 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.159343    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5758\" (UniqueName: \"kubernetes.io/projected/a6310814-479f-42c5-9eba-995181462a21-kube-api-access-m5758\") pod \"a6310814-479f-42c5-9eba-995181462a21\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") "
Jul 07 02:39:17.159599 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.159381    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6310814-479f-42c5-9eba-995181462a21-kube-rbac-proxy-sar-config\") pod \"a6310814-479f-42c5-9eba-995181462a21\" (UID: \"a6310814-479f-42c5-9eba-995181462a21\") "
Jul 07 02:39:17.159805 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.159782    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6310814-479f-42c5-9eba-995181462a21-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a6310814-479f-42c5-9eba-995181462a21" (UID: "a6310814-479f-42c5-9eba-995181462a21"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:39:17.161401 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.161369    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a6310814-479f-42c5-9eba-995181462a21" (UID: "a6310814-479f-42c5-9eba-995181462a21"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:39:17.161612 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.161592    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6310814-479f-42c5-9eba-995181462a21-kube-api-access-m5758" (OuterVolumeSpecName: "kube-api-access-m5758") pod "a6310814-479f-42c5-9eba-995181462a21" (UID: "a6310814-479f-42c5-9eba-995181462a21"). InnerVolumeSpecName "kube-api-access-m5758". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:39:17.260844 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.260807    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6310814-479f-42c5-9eba-995181462a21-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:39:17.260844 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.260833    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5758\" (UniqueName: \"kubernetes.io/projected/a6310814-479f-42c5-9eba-995181462a21-kube-api-access-m5758\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:39:17.260844 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.260843    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6310814-479f-42c5-9eba-995181462a21-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:39:17.797328 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.797294    2585 generic.go:356] "Generic (PLEG): container finished" podID="a6310814-479f-42c5-9eba-995181462a21" containerID="5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474" exitCode=0
Jul 07 02:39:17.797744 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.797351    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"
Jul 07 02:39:17.797744 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.797377    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" event={"ID":"a6310814-479f-42c5-9eba-995181462a21","Type":"ContainerDied","Data":"5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474"}
Jul 07 02:39:17.797744 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.797431    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd" event={"ID":"a6310814-479f-42c5-9eba-995181462a21","Type":"ContainerDied","Data":"38cbcfc5128c16c11fdcdfdf694eba95843563f4e13258c22c13b7e7b08bf078"}
Jul 07 02:39:17.797744 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.797477    2585 scope.go:117] "RemoveContainer" containerID="b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968"
Jul 07 02:39:17.797927 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.797821    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 07 02:39:17.806534 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.806517    2585 scope.go:117] "RemoveContainer" containerID="5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474"
Jul 07 02:39:17.813407 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.813390    2585 scope.go:117] "RemoveContainer" containerID="b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968"
Jul 07 02:39:17.813613 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:39:17.813598    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968\": container with ID starting with b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968 not found: ID does not exist" containerID="b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968"
Jul 07 02:39:17.813664 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.813621    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968"} err="failed to get container status \"b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968\": rpc error: code = NotFound desc = could not find container \"b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968\": container with ID starting with b1832a378192d5a36497a4ddc136260255e4867de97e63deebea8cb6e0c10968 not found: ID does not exist"
Jul 07 02:39:17.813664 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.813635    2585 scope.go:117] "RemoveContainer" containerID="5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474"
Jul 07 02:39:17.813881 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:39:17.813867    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474\": container with ID starting with 5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474 not found: ID does not exist" containerID="5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474"
Jul 07 02:39:17.813929 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.813882    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474"} err="failed to get container status \"5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474\": rpc error: code = NotFound desc = could not find container \"5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474\": container with ID starting with 5c19550a82e5998ae6c11a5f2528d79fa6aa26c565e78a6ccdeef09459e65474 not found: ID does not exist"
Jul 07 02:39:17.819773 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.819753    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"]
Jul 07 02:39:17.824779 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:17.824757    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cec47-predictor-6dccdfd5c6-ldwhd"]
Jul 07 02:39:18.397319 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:18.397287    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6310814-479f-42c5-9eba-995181462a21" path="/var/lib/kubelet/pods/a6310814-479f-42c5-9eba-995181462a21/volumes"
Jul 07 02:39:21.628552 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:21.628519    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:39:22.802878 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:22.802848    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:39:22.803287 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:22.803253    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 07 02:39:32.804115 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:32.804075    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 07 02:39:42.803873 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:42.803830    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 07 02:39:43.840064 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.840022    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"]
Jul 07 02:39:43.840433 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.840304    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container" containerID="cri-o://dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d" gracePeriod=30
Jul 07 02:39:43.840433 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.840399    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kube-rbac-proxy" containerID="cri-o://f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed" gracePeriod=30
Jul 07 02:39:43.950882 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.950849    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"]
Jul 07 02:39:43.951248 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.951228    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kube-rbac-proxy"
Jul 07 02:39:43.951335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.951252    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kube-rbac-proxy"
Jul 07 02:39:43.951335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.951301    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container"
Jul 07 02:39:43.951335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.951311    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container"
Jul 07 02:39:43.951504 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.951383    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kube-rbac-proxy"
Jul 07 02:39:43.951504 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.951396    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6310814-479f-42c5-9eba-995181462a21" containerName="kserve-container"
Jul 07 02:39:43.954628 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.954600    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:43.964245 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.964215    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7c238-predictor-serving-cert\""
Jul 07 02:39:43.966070 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.966054    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7c238-kube-rbac-proxy-sar-config\""
Jul 07 02:39:43.984855 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:43.984824    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"]
Jul 07 02:39:44.075863 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.075831    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1875431-a285-4261-9613-9843f0ece766-proxy-tls\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.076046 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.075886    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c1875431-a285-4261-9613-9843f0ece766-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.076046 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.075925    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz52b\" (UniqueName: \"kubernetes.io/projected/c1875431-a285-4261-9613-9843f0ece766-kube-api-access-cz52b\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.177487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.177377    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1875431-a285-4261-9613-9843f0ece766-proxy-tls\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.177487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.177428    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c1875431-a285-4261-9613-9843f0ece766-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.177487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.177479    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz52b\" (UniqueName: \"kubernetes.io/projected/c1875431-a285-4261-9613-9843f0ece766-kube-api-access-cz52b\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.178225 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.178151    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c1875431-a285-4261-9613-9843f0ece766-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.180019 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.179981    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1875431-a285-4261-9613-9843f0ece766-proxy-tls\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.185619 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.185598    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz52b\" (UniqueName: \"kubernetes.io/projected/c1875431-a285-4261-9613-9843f0ece766-kube-api-access-cz52b\") pod \"error-404-isvc-7c238-predictor-c4bd4d474-qb4ww\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") " pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.264424 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.264389    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.398598 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.398572    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"]
Jul 07 02:39:44.400414 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:39:44.400391    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1875431_a285_4261_9613_9843f0ece766.slice/crio-b35b28c203d369931b2ce9aa10db25698cc7da4bf42e36aaf82d419b8b39c4cd WatchSource:0}: Error finding container b35b28c203d369931b2ce9aa10db25698cc7da4bf42e36aaf82d419b8b39c4cd: Status 404 returned error can't find the container with id b35b28c203d369931b2ce9aa10db25698cc7da4bf42e36aaf82d419b8b39c4cd
Jul 07 02:39:44.911429 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.911349    2585 generic.go:356] "Generic (PLEG): container finished" podID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerID="f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed" exitCode=2
Jul 07 02:39:44.911429 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.911378    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" event={"ID":"86fa344a-3fb0-4d92-8c46-d3d4d24801a5","Type":"ContainerDied","Data":"f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed"}
Jul 07 02:39:44.913134 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.913105    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" event={"ID":"c1875431-a285-4261-9613-9843f0ece766","Type":"ContainerStarted","Data":"599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35"}
Jul 07 02:39:44.913258 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.913155    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" event={"ID":"c1875431-a285-4261-9613-9843f0ece766","Type":"ContainerStarted","Data":"e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac"}
Jul 07 02:39:44.913258 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.913178    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" event={"ID":"c1875431-a285-4261-9613-9843f0ece766","Type":"ContainerStarted","Data":"b35b28c203d369931b2ce9aa10db25698cc7da4bf42e36aaf82d419b8b39c4cd"}
Jul 07 02:39:44.913339 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.913327    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.913479 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.913465    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:44.914824 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.914801    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 07 02:39:44.935175 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:44.935129    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podStartSLOduration=1.9351133059999999 podStartE2EDuration="1.935113306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:39:44.933942527 +0000 UTC m=+1311.062029218" watchObservedRunningTime="2026-07-07 02:39:44.935113306 +0000 UTC m=+1311.063200061"
Jul 07 02:39:45.917785 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:45.917750    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 07 02:39:46.623121 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:46.623075    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Jul 07 02:39:47.094984 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.094960    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:39:47.103181 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.103149    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md6bb\" (UniqueName: \"kubernetes.io/projected/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-api-access-md6bb\") pod \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") "
Jul 07 02:39:47.103293 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.103197    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-rbac-proxy-sar-config\") pod \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") "
Jul 07 02:39:47.103293 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.103250    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls\") pod \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\" (UID: \"86fa344a-3fb0-4d92-8c46-d3d4d24801a5\") "
Jul 07 02:39:47.103629 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.103601    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "86fa344a-3fb0-4d92-8c46-d3d4d24801a5" (UID: "86fa344a-3fb0-4d92-8c46-d3d4d24801a5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:39:47.105371 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.105350    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "86fa344a-3fb0-4d92-8c46-d3d4d24801a5" (UID: "86fa344a-3fb0-4d92-8c46-d3d4d24801a5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:39:47.105371 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.105360    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-api-access-md6bb" (OuterVolumeSpecName: "kube-api-access-md6bb") pod "86fa344a-3fb0-4d92-8c46-d3d4d24801a5" (UID: "86fa344a-3fb0-4d92-8c46-d3d4d24801a5"). InnerVolumeSpecName "kube-api-access-md6bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:39:47.204243 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.204212    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md6bb\" (UniqueName: \"kubernetes.io/projected/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-api-access-md6bb\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:39:47.204243 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.204240    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:39:47.204243 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.204250    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86fa344a-3fb0-4d92-8c46-d3d4d24801a5-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:39:47.929611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.929575    2585 generic.go:356] "Generic (PLEG): container finished" podID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerID="dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d" exitCode=0
Jul 07 02:39:47.929882 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.929637    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"
Jul 07 02:39:47.929882 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.929647    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" event={"ID":"86fa344a-3fb0-4d92-8c46-d3d4d24801a5","Type":"ContainerDied","Data":"dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d"}
Jul 07 02:39:47.929882 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.929697    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825" event={"ID":"86fa344a-3fb0-4d92-8c46-d3d4d24801a5","Type":"ContainerDied","Data":"cc75620255fbee0a01baa17b35a48b6395f3d10433361ffe3ed78c01f2418cb2"}
Jul 07 02:39:47.929882 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.929723    2585 scope.go:117] "RemoveContainer" containerID="f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed"
Jul 07 02:39:47.943946 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.943924    2585 scope.go:117] "RemoveContainer" containerID="dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d"
Jul 07 02:39:47.951134 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.951116    2585 scope.go:117] "RemoveContainer" containerID="f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed"
Jul 07 02:39:47.951382 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:39:47.951364    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed\": container with ID starting with f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed not found: ID does not exist" containerID="f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed"
Jul 07 02:39:47.951426 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.951391    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed"} err="failed to get container status \"f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed\": rpc error: code = NotFound desc = could not find container \"f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed\": container with ID starting with f9c6c86c1f073135d925b54dd7d564d2a7d7db2f39abb6b2764b482431ce40ed not found: ID does not exist"
Jul 07 02:39:47.951426 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.951409    2585 scope.go:117] "RemoveContainer" containerID="dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d"
Jul 07 02:39:47.951666 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:39:47.951647    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d\": container with ID starting with dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d not found: ID does not exist" containerID="dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d"
Jul 07 02:39:47.951706 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.951674    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d"} err="failed to get container status \"dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d\": rpc error: code = NotFound desc = could not find container \"dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d\": container with ID starting with dd2efdf845b93f125b9c8f3dcf609edbd6697f6f9e5a228ad608582a2c2d042d not found: ID does not exist"
Jul 07 02:39:47.955855 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.955831    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"]
Jul 07 02:39:47.961492 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:47.961470    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-524cd-predictor-58fd54b564-2w825"]
Jul 07 02:39:48.397487 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:48.397432    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" path="/var/lib/kubelet/pods/86fa344a-3fb0-4d92-8c46-d3d4d24801a5/volumes"
Jul 07 02:39:50.923226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:50.923198    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:39:50.923767 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:50.923690    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 07 02:39:52.804132 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:39:52.804091    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jul 07 02:40:00.923770 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:40:00.923729    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 07 02:40:02.804610 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:40:02.804581    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:40:10.923633 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:40:10.923599    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 07 02:40:20.924021 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:40:20.923942    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 07 02:40:30.924615 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:40:30.924587    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:42:54.375328 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:42:54.375300    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:42:54.380871 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:42:54.380848    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:47:54.402108 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:47:54.402071    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:47:54.408515 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:47:54.408487    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:48:28.981980 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:28.981943    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"]
Jul 07 02:48:28.982602 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:28.982356    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container" containerID="cri-o://8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd" gracePeriod=30
Jul 07 02:48:28.983205 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:28.982804    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kube-rbac-proxy" containerID="cri-o://5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450" gracePeriod=30
Jul 07 02:48:29.034688 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.034647    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"]
Jul 07 02:48:29.035099 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.035085    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kube-rbac-proxy"
Jul 07 02:48:29.035146 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.035102    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kube-rbac-proxy"
Jul 07 02:48:29.035146 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.035124    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container"
Jul 07 02:48:29.035146 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.035133    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container"
Jul 07 02:48:29.035255 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.035228    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kserve-container"
Jul 07 02:48:29.035255 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.035242    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fa344a-3fb0-4d92-8c46-d3d4d24801a5" containerName="kube-rbac-proxy"
Jul 07 02:48:29.039051 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.039020    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.042337 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.042304    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4831c-kube-rbac-proxy-sar-config\""
Jul 07 02:48:29.042500 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.042310    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-4831c-predictor-serving-cert\""
Jul 07 02:48:29.048829 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.048806    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"]
Jul 07 02:48:29.099436 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.099388    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.099627 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.099440    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.099627 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.099564    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhd6k\" (UniqueName: \"kubernetes.io/projected/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-api-access-bhd6k\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.200007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.199960    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.200007 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.200006    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.200269 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.200047    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhd6k\" (UniqueName: \"kubernetes.io/projected/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-api-access-bhd6k\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.200269 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:48:29.200141    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-4831c-predictor-serving-cert: secret "error-404-isvc-4831c-predictor-serving-cert" not found
Jul 07 02:48:29.200269 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:48:29.200221    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls podName:5d5562c9-3add-4214-8263-1d68f69f9b48 nodeName:}" failed. No retries permitted until 2026-07-07 02:48:29.70019944 +0000 UTC m=+1835.828286123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls") pod "error-404-isvc-4831c-predictor-5944766686-tvxdv" (UID: "5d5562c9-3add-4214-8263-1d68f69f9b48") : secret "error-404-isvc-4831c-predictor-serving-cert" not found
Jul 07 02:48:29.200770 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.200752    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.209656 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.209622    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhd6k\" (UniqueName: \"kubernetes.io/projected/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-api-access-bhd6k\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.704401 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.704364    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.706891 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.706867    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls\") pod \"error-404-isvc-4831c-predictor-5944766686-tvxdv\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") " pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:29.952378 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:29.952336    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:30.040996 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:30.040965    2585 generic.go:356] "Generic (PLEG): container finished" podID="00124b43-38ed-4308-a2f1-e997b3a23856" containerID="5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450" exitCode=2
Jul 07 02:48:30.041356 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:30.041037    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" event={"ID":"00124b43-38ed-4308-a2f1-e997b3a23856","Type":"ContainerDied","Data":"5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450"}
Jul 07 02:48:30.092773 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:30.092561    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"]
Jul 07 02:48:30.095380 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:48:30.095353    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5562c9_3add_4214_8263_1d68f69f9b48.slice/crio-225e3b74f476b17ce9a2b1fd5d62c4b3b19240a10d914a7114345066b8b955f1 WatchSource:0}: Error finding container 225e3b74f476b17ce9a2b1fd5d62c4b3b19240a10d914a7114345066b8b955f1: Status 404 returned error can't find the container with id 225e3b74f476b17ce9a2b1fd5d62c4b3b19240a10d914a7114345066b8b955f1
Jul 07 02:48:31.047241 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:31.047204    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" event={"ID":"5d5562c9-3add-4214-8263-1d68f69f9b48","Type":"ContainerStarted","Data":"8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61"}
Jul 07 02:48:31.047702 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:31.047269    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" event={"ID":"5d5562c9-3add-4214-8263-1d68f69f9b48","Type":"ContainerStarted","Data":"1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144"}
Jul 07 02:48:31.047702 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:31.047303    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:31.047702 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:31.047333    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" event={"ID":"5d5562c9-3add-4214-8263-1d68f69f9b48","Type":"ContainerStarted","Data":"225e3b74f476b17ce9a2b1fd5d62c4b3b19240a10d914a7114345066b8b955f1"}
Jul 07 02:48:31.047702 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:31.047368    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:31.048878 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:31.048853    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 07 02:48:31.067136 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:31.067077    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podStartSLOduration=2.06706175 podStartE2EDuration="2.06706175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:48:31.065793335 +0000 UTC m=+1837.193880027" watchObservedRunningTime="2026-07-07 02:48:31.06706175 +0000 UTC m=+1837.195148442"
Jul 07 02:48:32.051644 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.051601    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 07 02:48:32.439810 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.439784    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:48:32.525477 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.525417    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls\") pod \"00124b43-38ed-4308-a2f1-e997b3a23856\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") "
Jul 07 02:48:32.525477 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.525482    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00124b43-38ed-4308-a2f1-e997b3a23856-kube-rbac-proxy-sar-config\") pod \"00124b43-38ed-4308-a2f1-e997b3a23856\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") "
Jul 07 02:48:32.525682 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.525510    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngkx2\" (UniqueName: \"kubernetes.io/projected/00124b43-38ed-4308-a2f1-e997b3a23856-kube-api-access-ngkx2\") pod \"00124b43-38ed-4308-a2f1-e997b3a23856\" (UID: \"00124b43-38ed-4308-a2f1-e997b3a23856\") "
Jul 07 02:48:32.525910 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.525881    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00124b43-38ed-4308-a2f1-e997b3a23856-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "00124b43-38ed-4308-a2f1-e997b3a23856" (UID: "00124b43-38ed-4308-a2f1-e997b3a23856"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:48:32.527800 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.527765    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "00124b43-38ed-4308-a2f1-e997b3a23856" (UID: "00124b43-38ed-4308-a2f1-e997b3a23856"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:48:32.527928 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.527833    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00124b43-38ed-4308-a2f1-e997b3a23856-kube-api-access-ngkx2" (OuterVolumeSpecName: "kube-api-access-ngkx2") pod "00124b43-38ed-4308-a2f1-e997b3a23856" (UID: "00124b43-38ed-4308-a2f1-e997b3a23856"). InnerVolumeSpecName "kube-api-access-ngkx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:48:32.626630 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.626529    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00124b43-38ed-4308-a2f1-e997b3a23856-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:48:32.626630 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.626576    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00124b43-38ed-4308-a2f1-e997b3a23856-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:48:32.626630 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:32.626595    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngkx2\" (UniqueName: \"kubernetes.io/projected/00124b43-38ed-4308-a2f1-e997b3a23856-kube-api-access-ngkx2\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:48:33.057458 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.057411    2585 generic.go:356] "Generic (PLEG): container finished" podID="00124b43-38ed-4308-a2f1-e997b3a23856" containerID="8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd" exitCode=0
Jul 07 02:48:33.057913 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.057478    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"
Jul 07 02:48:33.057913 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.057502    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" event={"ID":"00124b43-38ed-4308-a2f1-e997b3a23856","Type":"ContainerDied","Data":"8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd"}
Jul 07 02:48:33.057913 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.057582    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg" event={"ID":"00124b43-38ed-4308-a2f1-e997b3a23856","Type":"ContainerDied","Data":"f2f064adc1f0c44e935cf8d0eb36957e381337a17055fce48f87a8888686c41b"}
Jul 07 02:48:33.057913 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.057628    2585 scope.go:117] "RemoveContainer" containerID="5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450"
Jul 07 02:48:33.069639 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.069611    2585 scope.go:117] "RemoveContainer" containerID="8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd"
Jul 07 02:48:33.078121 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.078100    2585 scope.go:117] "RemoveContainer" containerID="5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450"
Jul 07 02:48:33.078383 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:48:33.078363    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450\": container with ID starting with 5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450 not found: ID does not exist" containerID="5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450"
Jul 07 02:48:33.078433 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.078392    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450"} err="failed to get container status \"5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450\": rpc error: code = NotFound desc = could not find container \"5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450\": container with ID starting with 5630c4816bfffe141f0a56c08ddb2e68948ec90fa237ce565db1386216da6450 not found: ID does not exist"
Jul 07 02:48:33.078433 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.078410    2585 scope.go:117] "RemoveContainer" containerID="8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd"
Jul 07 02:48:33.078666 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:48:33.078650    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd\": container with ID starting with 8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd not found: ID does not exist" containerID="8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd"
Jul 07 02:48:33.078706 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.078670    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd"} err="failed to get container status \"8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd\": rpc error: code = NotFound desc = could not find container \"8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd\": container with ID starting with 8c45dc228622d2574740dccf1d5062f47ca36cde3cd8b58e3a61630470a493cd not found: ID does not exist"
Jul 07 02:48:33.085735 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.085710    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"]
Jul 07 02:48:33.090419 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:33.090397    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-e4a8c-predictor-669cdb968b-nfffg"]
Jul 07 02:48:34.397209 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:34.397173    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" path="/var/lib/kubelet/pods/00124b43-38ed-4308-a2f1-e997b3a23856/volumes"
Jul 07 02:48:37.055738 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:37.055712    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:48:37.056220 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:37.056159    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 07 02:48:47.056708 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:47.056666    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 07 02:48:57.056396 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:57.056348    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 07 02:48:58.836022 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.835987    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"]
Jul 07 02:48:58.836466 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.836225    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" containerID="cri-o://e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac" gracePeriod=30
Jul 07 02:48:58.836466 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.836302    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kube-rbac-proxy" containerID="cri-o://599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35" gracePeriod=30
Jul 07 02:48:58.962563 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.962526    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"]
Jul 07 02:48:58.962998 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.962982    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kube-rbac-proxy"
Jul 07 02:48:58.963083 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.963000    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kube-rbac-proxy"
Jul 07 02:48:58.963083 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.963020    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container"
Jul 07 02:48:58.963083 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.963031    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container"
Jul 07 02:48:58.963231 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.963107    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kserve-container"
Jul 07 02:48:58.963231 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.963118    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="00124b43-38ed-4308-a2f1-e997b3a23856" containerName="kube-rbac-proxy"
Jul 07 02:48:58.966291 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.966263    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:58.969762 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.969731    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b2091-predictor-serving-cert\""
Jul 07 02:48:58.969893 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.969732    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b2091-kube-rbac-proxy-sar-config\""
Jul 07 02:48:58.984689 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:58.984660    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"]
Jul 07 02:48:59.045464 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.045408    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.045658 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.045561    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.045658 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.045613    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nzxw\" (UniqueName: \"kubernetes.io/projected/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-api-access-4nzxw\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.146625 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.146526    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.146625 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.146575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzxw\" (UniqueName: \"kubernetes.io/projected/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-api-access-4nzxw\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.146841 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.146628    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.146841 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:48:59.146690    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-b2091-predictor-serving-cert: secret "error-404-isvc-b2091-predictor-serving-cert" not found
Jul 07 02:48:59.146841 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:48:59.146762    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls podName:44124dcf-f193-492c-a991-7f6072bd3fb0 nodeName:}" failed. No retries permitted until 2026-07-07 02:48:59.646738262 +0000 UTC m=+1865.774824932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls") pod "error-404-isvc-b2091-predictor-565f465b48-pprvw" (UID: "44124dcf-f193-492c-a991-7f6072bd3fb0") : secret "error-404-isvc-b2091-predictor-serving-cert" not found
Jul 07 02:48:59.147327 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.147308    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.169885 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.169849    2585 generic.go:356] "Generic (PLEG): container finished" podID="c1875431-a285-4261-9613-9843f0ece766" containerID="599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35" exitCode=2
Jul 07 02:48:59.170056 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.169974    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" event={"ID":"c1875431-a285-4261-9613-9843f0ece766","Type":"ContainerDied","Data":"599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35"}
Jul 07 02:48:59.170056 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.170041    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nzxw\" (UniqueName: \"kubernetes.io/projected/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-api-access-4nzxw\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.649559 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.649520    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.652056 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.652036    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls\") pod \"error-404-isvc-b2091-predictor-565f465b48-pprvw\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") " pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:48:59.878226 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:48:59.878171    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:49:00.022794 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.022752    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"]
Jul 07 02:49:00.025101 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:49:00.025064    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44124dcf_f193_492c_a991_7f6072bd3fb0.slice/crio-815c889d7f2ec41df3e6c309ab6f159bde93f97b9d95ad2da21fac96cedbb04c WatchSource:0}: Error finding container 815c889d7f2ec41df3e6c309ab6f159bde93f97b9d95ad2da21fac96cedbb04c: Status 404 returned error can't find the container with id 815c889d7f2ec41df3e6c309ab6f159bde93f97b9d95ad2da21fac96cedbb04c
Jul 07 02:49:00.176273 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.176242    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" event={"ID":"44124dcf-f193-492c-a991-7f6072bd3fb0","Type":"ContainerStarted","Data":"03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324"}
Jul 07 02:49:00.176508 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.176304    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" event={"ID":"44124dcf-f193-492c-a991-7f6072bd3fb0","Type":"ContainerStarted","Data":"7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653"}
Jul 07 02:49:00.176508 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.176331    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" event={"ID":"44124dcf-f193-492c-a991-7f6072bd3fb0","Type":"ContainerStarted","Data":"815c889d7f2ec41df3e6c309ab6f159bde93f97b9d95ad2da21fac96cedbb04c"}
Jul 07 02:49:00.176508 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.176357    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:49:00.202927 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.202869    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podStartSLOduration=2.202851256 podStartE2EDuration="2.202851256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:49:00.202419602 +0000 UTC m=+1866.330506294" watchObservedRunningTime="2026-07-07 02:49:00.202851256 +0000 UTC m=+1866.330938010"
Jul 07 02:49:00.918900 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.918856    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Jul 07 02:49:00.924055 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:00.924018    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 07 02:49:01.180071 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:01.179977    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:49:01.181461 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:01.181420    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 07 02:49:02.184900 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.184855    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 07 02:49:02.394333 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.394300    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:49:02.474177 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.474073    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1875431-a285-4261-9613-9843f0ece766-proxy-tls\") pod \"c1875431-a285-4261-9613-9843f0ece766\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") "
Jul 07 02:49:02.474177 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.474139    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz52b\" (UniqueName: \"kubernetes.io/projected/c1875431-a285-4261-9613-9843f0ece766-kube-api-access-cz52b\") pod \"c1875431-a285-4261-9613-9843f0ece766\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") "
Jul 07 02:49:02.474411 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.474238    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c1875431-a285-4261-9613-9843f0ece766-kube-rbac-proxy-sar-config\") pod \"c1875431-a285-4261-9613-9843f0ece766\" (UID: \"c1875431-a285-4261-9613-9843f0ece766\") "
Jul 07 02:49:02.474640 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.474612    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1875431-a285-4261-9613-9843f0ece766-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c1875431-a285-4261-9613-9843f0ece766" (UID: "c1875431-a285-4261-9613-9843f0ece766"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:49:02.476403 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.476385    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1875431-a285-4261-9613-9843f0ece766-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c1875431-a285-4261-9613-9843f0ece766" (UID: "c1875431-a285-4261-9613-9843f0ece766"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:49:02.476489 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.476406    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1875431-a285-4261-9613-9843f0ece766-kube-api-access-cz52b" (OuterVolumeSpecName: "kube-api-access-cz52b") pod "c1875431-a285-4261-9613-9843f0ece766" (UID: "c1875431-a285-4261-9613-9843f0ece766"). InnerVolumeSpecName "kube-api-access-cz52b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:49:02.575254 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.575216    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c1875431-a285-4261-9613-9843f0ece766-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:49:02.575254 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.575246    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1875431-a285-4261-9613-9843f0ece766-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:49:02.575254 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:02.575256    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz52b\" (UniqueName: \"kubernetes.io/projected/c1875431-a285-4261-9613-9843f0ece766-kube-api-access-cz52b\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:49:03.191013 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.190977    2585 generic.go:356] "Generic (PLEG): container finished" podID="c1875431-a285-4261-9613-9843f0ece766" containerID="e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac" exitCode=0
Jul 07 02:49:03.191514 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.191035    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"
Jul 07 02:49:03.191514 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.191041    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" event={"ID":"c1875431-a285-4261-9613-9843f0ece766","Type":"ContainerDied","Data":"e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac"}
Jul 07 02:49:03.191514 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.191105    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww" event={"ID":"c1875431-a285-4261-9613-9843f0ece766","Type":"ContainerDied","Data":"b35b28c203d369931b2ce9aa10db25698cc7da4bf42e36aaf82d419b8b39c4cd"}
Jul 07 02:49:03.191514 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.191147    2585 scope.go:117] "RemoveContainer" containerID="599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35"
Jul 07 02:49:03.199638 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.199618    2585 scope.go:117] "RemoveContainer" containerID="e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac"
Jul 07 02:49:03.207189 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.207169    2585 scope.go:117] "RemoveContainer" containerID="599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35"
Jul 07 02:49:03.207478 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:49:03.207461    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35\": container with ID starting with 599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35 not found: ID does not exist" containerID="599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35"
Jul 07 02:49:03.207533 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.207486    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35"} err="failed to get container status \"599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35\": rpc error: code = NotFound desc = could not find container \"599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35\": container with ID starting with 599f935205de75c9153ffb8134163d879530e3c9812c07f1198c2286ec177a35 not found: ID does not exist"
Jul 07 02:49:03.207533 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.207503    2585 scope.go:117] "RemoveContainer" containerID="e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac"
Jul 07 02:49:03.207724 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:49:03.207709    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac\": container with ID starting with e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac not found: ID does not exist" containerID="e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac"
Jul 07 02:49:03.207769 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.207725    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac"} err="failed to get container status \"e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac\": rpc error: code = NotFound desc = could not find container \"e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac\": container with ID starting with e06f879fcc82fbabc2515f0ece59b4de721c4eceb5ca45647f208ebdd8f2edac not found: ID does not exist"
Jul 07 02:49:03.212783 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.212759    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"]
Jul 07 02:49:03.219208 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:03.219177    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7c238-predictor-c4bd4d474-qb4ww"]
Jul 07 02:49:04.398078 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:04.398043    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1875431-a285-4261-9613-9843f0ece766" path="/var/lib/kubelet/pods/c1875431-a285-4261-9613-9843f0ece766/volumes"
Jul 07 02:49:07.056519 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:07.056477    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 07 02:49:07.189588 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:07.189559    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:49:07.190012 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:07.189988    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 07 02:49:17.057568 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:17.057491    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:49:17.190009 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:17.189968    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 07 02:49:27.190210 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:27.190170    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 07 02:49:37.190199 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:37.190146    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 07 02:49:47.191760 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:47.191731    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 02:49:49.282729 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.282691    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"]
Jul 07 02:49:49.283199 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.283005    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container" containerID="cri-o://1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144" gracePeriod=30
Jul 07 02:49:49.283199 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.283047    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kube-rbac-proxy" containerID="cri-o://8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61" gracePeriod=30
Jul 07 02:49:49.412441 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.412403    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"]
Jul 07 02:49:49.412879 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.412863    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container"
Jul 07 02:49:49.412944 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.412882    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container"
Jul 07 02:49:49.412944 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.412899    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kube-rbac-proxy"
Jul 07 02:49:49.412944 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.412909    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kube-rbac-proxy"
Jul 07 02:49:49.413091 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.413000    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kube-rbac-proxy"
Jul 07 02:49:49.413091 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.413015    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1875431-a285-4261-9613-9843f0ece766" containerName="kserve-container"
Jul 07 02:49:49.416228 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.416209    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.429277 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.429253    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8f974-predictor-serving-cert\""
Jul 07 02:49:49.429399 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.429378    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8f974-kube-rbac-proxy-sar-config\""
Jul 07 02:49:49.444661 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.444633    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"]
Jul 07 02:49:49.551315 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.551233    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.551315 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.551263    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.551315 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.551282    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw2fm\" (UniqueName: \"kubernetes.io/projected/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-api-access-tw2fm\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.651813 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.651769    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.651813 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.651814    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.652101 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.651842    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw2fm\" (UniqueName: \"kubernetes.io/projected/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-api-access-tw2fm\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.652101 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:49:49.651927    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-8f974-predictor-serving-cert: secret "error-404-isvc-8f974-predictor-serving-cert" not found
Jul 07 02:49:49.652101 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:49:49.652014    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls podName:7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44 nodeName:}" failed. No retries permitted until 2026-07-07 02:49:50.151991715 +0000 UTC m=+1916.280078384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls") pod "error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" (UID: "7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44") : secret "error-404-isvc-8f974-predictor-serving-cert" not found
Jul 07 02:49:49.652467 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.652427    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:49.661153 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:49.661127    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw2fm\" (UniqueName: \"kubernetes.io/projected/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-api-access-tw2fm\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:50.156302 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:50.156265    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:50.158890 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:50.158858    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls\") pod \"error-404-isvc-8f974-predictor-854b47f8fb-ljkwj\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") " pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:50.328567 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:50.328513    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:50.391916 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:50.391873    2585 generic.go:356] "Generic (PLEG): container finished" podID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerID="8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61" exitCode=2
Jul 07 02:49:50.399096 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:50.399062    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" event={"ID":"5d5562c9-3add-4214-8263-1d68f69f9b48","Type":"ContainerDied","Data":"8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61"}
Jul 07 02:49:50.466891 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:50.466862    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"]
Jul 07 02:49:50.469437 ip-10-0-142-230 kubenswrapper[2585]: W0707 02:49:50.469410    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce7b8ee_a3c6_4422_b8c0_cb655b0e8a44.slice/crio-7860a465a08220087d126ae8a0fce989fee43f05e0dcad0afd49b884c4e08f47 WatchSource:0}: Error finding container 7860a465a08220087d126ae8a0fce989fee43f05e0dcad0afd49b884c4e08f47: Status 404 returned error can't find the container with id 7860a465a08220087d126ae8a0fce989fee43f05e0dcad0afd49b884c4e08f47
Jul 07 02:49:51.397882 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:51.397847    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" event={"ID":"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44","Type":"ContainerStarted","Data":"16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05"}
Jul 07 02:49:51.398335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:51.397920    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:51.398335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:51.398041    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:51.398335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:51.398072    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" event={"ID":"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44","Type":"ContainerStarted","Data":"92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d"}
Jul 07 02:49:51.398335 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:51.398102    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" event={"ID":"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44","Type":"ContainerStarted","Data":"7860a465a08220087d126ae8a0fce989fee43f05e0dcad0afd49b884c4e08f47"}
Jul 07 02:49:51.399545 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:51.399516    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 07 02:49:51.418754 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:51.418709    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podStartSLOduration=2.418692785 podStartE2EDuration="2.418692785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:49:51.417343495 +0000 UTC m=+1917.545430187" watchObservedRunningTime="2026-07-07 02:49:51.418692785 +0000 UTC m=+1917.546779475"
Jul 07 02:49:52.052294 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.052247    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused"
Jul 07 02:49:52.401697 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.401604    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 07 02:49:52.639946 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.639919    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:49:52.781204 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.781166    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-rbac-proxy-sar-config\") pod \"5d5562c9-3add-4214-8263-1d68f69f9b48\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") "
Jul 07 02:49:52.781384 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.781249    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls\") pod \"5d5562c9-3add-4214-8263-1d68f69f9b48\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") "
Jul 07 02:49:52.781384 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.781350    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhd6k\" (UniqueName: \"kubernetes.io/projected/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-api-access-bhd6k\") pod \"5d5562c9-3add-4214-8263-1d68f69f9b48\" (UID: \"5d5562c9-3add-4214-8263-1d68f69f9b48\") "
Jul 07 02:49:52.781583 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.781561    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5d5562c9-3add-4214-8263-1d68f69f9b48" (UID: "5d5562c9-3add-4214-8263-1d68f69f9b48"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:49:52.783522 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.783499    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5d5562c9-3add-4214-8263-1d68f69f9b48" (UID: "5d5562c9-3add-4214-8263-1d68f69f9b48"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:49:52.783653 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.783630    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-api-access-bhd6k" (OuterVolumeSpecName: "kube-api-access-bhd6k") pod "5d5562c9-3add-4214-8263-1d68f69f9b48" (UID: "5d5562c9-3add-4214-8263-1d68f69f9b48"). InnerVolumeSpecName "kube-api-access-bhd6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:49:52.882611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.882573    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5d5562c9-3add-4214-8263-1d68f69f9b48-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:49:52.882611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.882605    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhd6k\" (UniqueName: \"kubernetes.io/projected/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-api-access-bhd6k\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:49:52.882611 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:52.882615    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5d5562c9-3add-4214-8263-1d68f69f9b48-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:49:53.406826 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.406791    2585 generic.go:356] "Generic (PLEG): container finished" podID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerID="1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144" exitCode=0
Jul 07 02:49:53.407280 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.406843    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"
Jul 07 02:49:53.407280 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.406869    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" event={"ID":"5d5562c9-3add-4214-8263-1d68f69f9b48","Type":"ContainerDied","Data":"1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144"}
Jul 07 02:49:53.407280 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.406932    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv" event={"ID":"5d5562c9-3add-4214-8263-1d68f69f9b48","Type":"ContainerDied","Data":"225e3b74f476b17ce9a2b1fd5d62c4b3b19240a10d914a7114345066b8b955f1"}
Jul 07 02:49:53.407280 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.406975    2585 scope.go:117] "RemoveContainer" containerID="8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61"
Jul 07 02:49:53.415985 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.415965    2585 scope.go:117] "RemoveContainer" containerID="1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144"
Jul 07 02:49:53.424084 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.424063    2585 scope.go:117] "RemoveContainer" containerID="8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61"
Jul 07 02:49:53.424378 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:49:53.424358    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61\": container with ID starting with 8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61 not found: ID does not exist" containerID="8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61"
Jul 07 02:49:53.424466 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.424388    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61"} err="failed to get container status \"8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61\": rpc error: code = NotFound desc = could not find container \"8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61\": container with ID starting with 8721885a3798706e1f680b42343bdeb083ceead7606a7cde59d1092ca936cb61 not found: ID does not exist"
Jul 07 02:49:53.424466 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.424406    2585 scope.go:117] "RemoveContainer" containerID="1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144"
Jul 07 02:49:53.424706 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:49:53.424686    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144\": container with ID starting with 1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144 not found: ID does not exist" containerID="1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144"
Jul 07 02:49:53.424747 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.424715    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144"} err="failed to get container status \"1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144\": rpc error: code = NotFound desc = could not find container \"1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144\": container with ID starting with 1ec61b2652c9c4621a91b8f79a3753a0a645451ce1adb118d672f69600a1b144 not found: ID does not exist"
Jul 07 02:49:53.429666 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.429645    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"]
Jul 07 02:49:53.433009 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:53.432988    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-4831c-predictor-5944766686-tvxdv"]
Jul 07 02:49:54.397326 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:54.397287    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" path="/var/lib/kubelet/pods/5d5562c9-3add-4214-8263-1d68f69f9b48/volumes"
Jul 07 02:49:57.405825 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:57.405798    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:49:57.406273 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:49:57.406248    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 07 02:50:07.406241 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:50:07.406201    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 07 02:50:17.406286 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:50:17.406243    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 07 02:50:27.406397 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:50:27.406354    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 07 02:50:37.408272 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:50:37.408241    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:52:54.430990 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:52:54.430950    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:52:54.437389 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:52:54.437365    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:57:54.453959 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:57:54.453930    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:57:54.460202 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:57:54.460179    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 02:59:04.140654 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:04.140619    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"]
Jul 07 02:59:04.141201 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:04.140849    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" containerID="cri-o://92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d" gracePeriod=30
Jul 07 02:59:04.141201 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:04.140955    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kube-rbac-proxy" containerID="cri-o://16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05" gracePeriod=30
Jul 07 02:59:04.691034 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:04.691000    2585 generic.go:356] "Generic (PLEG): container finished" podID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerID="16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05" exitCode=2
Jul 07 02:59:04.691208 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:04.691054    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" event={"ID":"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44","Type":"ContainerDied","Data":"16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05"}
Jul 07 02:59:07.402221 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.402176    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Jul 07 02:59:07.483997 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.483968    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:59:07.659477 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.659352    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls\") pod \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") "
Jul 07 02:59:07.659477 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.659402    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-rbac-proxy-sar-config\") pod \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") "
Jul 07 02:59:07.659477 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.659437    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw2fm\" (UniqueName: \"kubernetes.io/projected/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-api-access-tw2fm\") pod \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\" (UID: \"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44\") "
Jul 07 02:59:07.659852 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.659825    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" (UID: "7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 02:59:07.661653 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.661635    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" (UID: "7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 02:59:07.661773 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.661757    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-api-access-tw2fm" (OuterVolumeSpecName: "kube-api-access-tw2fm") pod "7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" (UID: "7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44"). InnerVolumeSpecName "kube-api-access-tw2fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 02:59:07.705064 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.705027    2585 generic.go:356] "Generic (PLEG): container finished" podID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerID="92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d" exitCode=0
Jul 07 02:59:07.705269 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.705073    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"
Jul 07 02:59:07.705269 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.705100    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" event={"ID":"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44","Type":"ContainerDied","Data":"92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d"}
Jul 07 02:59:07.705269 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.705161    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" event={"ID":"7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44","Type":"ContainerDied","Data":"7860a465a08220087d126ae8a0fce989fee43f05e0dcad0afd49b884c4e08f47"}
Jul 07 02:59:07.705269 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.705196    2585 scope.go:117] "RemoveContainer" containerID="16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05"
Jul 07 02:59:07.716425 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.716405    2585 scope.go:117] "RemoveContainer" containerID="92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d"
Jul 07 02:59:07.724241 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.724219    2585 scope.go:117] "RemoveContainer" containerID="16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05"
Jul 07 02:59:07.724620 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:59:07.724598    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05\": container with ID starting with 16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05 not found: ID does not exist" containerID="16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05"
Jul 07 02:59:07.724678 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.724631    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05"} err="failed to get container status \"16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05\": rpc error: code = NotFound desc = could not find container \"16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05\": container with ID starting with 16609a89a8e033403535ccdbd538db0068bdbb62db104c211fbe8859f153ad05 not found: ID does not exist"
Jul 07 02:59:07.724678 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.724650    2585 scope.go:117] "RemoveContainer" containerID="92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d"
Jul 07 02:59:07.724902 ip-10-0-142-230 kubenswrapper[2585]: E0707 02:59:07.724881    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d\": container with ID starting with 92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d not found: ID does not exist" containerID="92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d"
Jul 07 02:59:07.724947 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.724909    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d"} err="failed to get container status \"92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d\": rpc error: code = NotFound desc = could not find container \"92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d\": container with ID starting with 92461bbb225ab9d3e20c6e06ba53ee6d163320859d2314dfe9858c4fd650d80d not found: ID does not exist"
Jul 07 02:59:07.734313 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.734283    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"]
Jul 07 02:59:07.736685 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.736660    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj"]
Jul 07 02:59:07.760735 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.760705    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:59:07.760735 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.760734    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:59:07.760870 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:07.760744    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw2fm\" (UniqueName: \"kubernetes.io/projected/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44-kube-api-access-tw2fm\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 02:59:08.397065 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:08.397033    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" path="/var/lib/kubelet/pods/7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44/volumes"
Jul 07 02:59:08.407330 ip-10-0-142-230 kubenswrapper[2585]: I0707 02:59:08.407296    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8f974-predictor-854b47f8fb-ljkwj" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: i/o timeout"
Jul 07 03:02:54.475947 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:02:54.475833    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 03:02:54.483425 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:02:54.483404    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 03:06:18.250190 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:18.250086    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"]
Jul 07 03:06:18.250932 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:18.250878    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container" containerID="cri-o://7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653" gracePeriod=30
Jul 07 03:06:18.251127 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:18.251107    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kube-rbac-proxy" containerID="cri-o://03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324" gracePeriod=30
Jul 07 03:06:18.430746 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:18.430711    2585 generic.go:356] "Generic (PLEG): container finished" podID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerID="03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324" exitCode=2
Jul 07 03:06:18.430914 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:18.430781    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" event={"ID":"44124dcf-f193-492c-a991-7f6072bd3fb0","Type":"ContainerDied","Data":"03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324"}
Jul 07 03:06:21.209135 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.209110    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 03:06:21.367606 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.367493    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-rbac-proxy-sar-config\") pod \"44124dcf-f193-492c-a991-7f6072bd3fb0\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") "
Jul 07 03:06:21.367606 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.367550    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls\") pod \"44124dcf-f193-492c-a991-7f6072bd3fb0\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") "
Jul 07 03:06:21.367606 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.367581    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nzxw\" (UniqueName: \"kubernetes.io/projected/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-api-access-4nzxw\") pod \"44124dcf-f193-492c-a991-7f6072bd3fb0\" (UID: \"44124dcf-f193-492c-a991-7f6072bd3fb0\") "
Jul 07 03:06:21.367872 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.367850    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "44124dcf-f193-492c-a991-7f6072bd3fb0" (UID: "44124dcf-f193-492c-a991-7f6072bd3fb0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 03:06:21.369749 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.369718    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-api-access-4nzxw" (OuterVolumeSpecName: "kube-api-access-4nzxw") pod "44124dcf-f193-492c-a991-7f6072bd3fb0" (UID: "44124dcf-f193-492c-a991-7f6072bd3fb0"). InnerVolumeSpecName "kube-api-access-4nzxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 03:06:21.369847 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.369759    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "44124dcf-f193-492c-a991-7f6072bd3fb0" (UID: "44124dcf-f193-492c-a991-7f6072bd3fb0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 03:06:21.444203 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.444160    2585 generic.go:356] "Generic (PLEG): container finished" podID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerID="7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653" exitCode=0
Jul 07 03:06:21.444203 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.444205    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" event={"ID":"44124dcf-f193-492c-a991-7f6072bd3fb0","Type":"ContainerDied","Data":"7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653"}
Jul 07 03:06:21.444396 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.444215    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"
Jul 07 03:06:21.444396 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.444255    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw" event={"ID":"44124dcf-f193-492c-a991-7f6072bd3fb0","Type":"ContainerDied","Data":"815c889d7f2ec41df3e6c309ab6f159bde93f97b9d95ad2da21fac96cedbb04c"}
Jul 07 03:06:21.444396 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.444279    2585 scope.go:117] "RemoveContainer" containerID="03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324"
Jul 07 03:06:21.452836 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.452818    2585 scope.go:117] "RemoveContainer" containerID="7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653"
Jul 07 03:06:21.460370 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.460351    2585 scope.go:117] "RemoveContainer" containerID="03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324"
Jul 07 03:06:21.460633 ip-10-0-142-230 kubenswrapper[2585]: E0707 03:06:21.460612    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324\": container with ID starting with 03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324 not found: ID does not exist" containerID="03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324"
Jul 07 03:06:21.460678 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.460643    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324"} err="failed to get container status \"03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324\": rpc error: code = NotFound desc = could not find container \"03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324\": container with ID starting with 03feadd4afce6042050337b8647c1f68722b2deb96f4cf77897616f0ff621324 not found: ID does not exist"
Jul 07 03:06:21.460678 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.460666    2585 scope.go:117] "RemoveContainer" containerID="7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653"
Jul 07 03:06:21.460901 ip-10-0-142-230 kubenswrapper[2585]: E0707 03:06:21.460884    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653\": container with ID starting with 7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653 not found: ID does not exist" containerID="7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653"
Jul 07 03:06:21.460936 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.460908    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653"} err="failed to get container status \"7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653\": rpc error: code = NotFound desc = could not find container \"7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653\": container with ID starting with 7af687887d6d6fe1a653087210fb8197a190bce3042d876e1efec68eaab75653 not found: ID does not exist"
Jul 07 03:06:21.465472 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.465430    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"]
Jul 07 03:06:21.468934 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.468911    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 03:06:21.468934 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.468934    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44124dcf-f193-492c-a991-7f6072bd3fb0-proxy-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 03:06:21.469088 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.468945    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nzxw\" (UniqueName: \"kubernetes.io/projected/44124dcf-f193-492c-a991-7f6072bd3fb0-kube-api-access-4nzxw\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\""
Jul 07 03:06:21.469088 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:21.468955    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b2091-predictor-565f465b48-pprvw"]
Jul 07 03:06:22.397054 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:22.397008    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" path="/var/lib/kubelet/pods/44124dcf-f193-492c-a991-7f6072bd3fb0/volumes"
Jul 07 03:06:52.472071 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:52.472033    2585 ???:1] "http: TLS handshake error from 10.0.138.74:40616: EOF"
Jul 07 03:06:52.479389 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:52.479367    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n82t2_25e0d312-fbb7-40bd-93b3-ce1dd410426f/global-pull-secret-syncer/0.log"
Jul 07 03:06:52.656145 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:52.656113    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rkw64_38bb8a4b-467c-44c3-b2f0-01095131796a/konnectivity-agent/0.log"
Jul 07 03:06:52.733136 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:52.733056    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-230.ec2.internal_61aad07ba0bedc3a4e130d11147f6dd3/haproxy/0.log"
Jul 07 03:06:57.313145 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:57.313101    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vbq2k_fd035722-e83d-4655-a185-6491fe845cc0/node-exporter/0.log"
Jul 07 03:06:57.341130 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:57.341100    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vbq2k_fd035722-e83d-4655-a185-6491fe845cc0/kube-rbac-proxy/0.log"
Jul 07 03:06:57.369861 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:57.369835    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vbq2k_fd035722-e83d-4655-a185-6491fe845cc0/init-textfile/0.log"
Jul 07 03:06:59.392944 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:06:59.392916    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-tr6tr_f0eb21a1-d159-4e50-8f8a-548cca607197/networking-console-plugin/0.log"
Jul 07 03:07:00.343483 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.343421    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-44thl_0650f4f3-6638-486c-b6a8-92eff0dd11a1/download-server/0.log"
Jul 07 03:07:00.858330 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858297    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"]
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858620    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858632    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858642    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kube-rbac-proxy"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858648    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kube-rbac-proxy"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858657    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kube-rbac-proxy"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858663    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kube-rbac-proxy"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858669    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kube-rbac-proxy"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858674    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kube-rbac-proxy"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858682    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858687    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858698    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858703    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container"
Jul 07 03:07:00.858744 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858748    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kserve-container"
Jul 07 03:07:00.859116 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858754    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kserve-container"
Jul 07 03:07:00.859116 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858760    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kserve-container"
Jul 07 03:07:00.859116 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858767    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5562c9-3add-4214-8263-1d68f69f9b48" containerName="kube-rbac-proxy"
Jul 07 03:07:00.859116 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858772    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce7b8ee-a3c6-4422-b8c0-cb655b0e8a44" containerName="kube-rbac-proxy"
Jul 07 03:07:00.859116 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.858778    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="44124dcf-f193-492c-a991-7f6072bd3fb0" containerName="kube-rbac-proxy"
Jul 07 03:07:00.862828 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.862807    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:00.865603 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.865574    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h4pjz\"/\"kube-root-ca.crt\""
Jul 07 03:07:00.865914 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.865608    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h4pjz\"/\"openshift-service-ca.crt\""
Jul 07 03:07:00.866895 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.866879    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h4pjz\"/\"default-dockercfg-5lngh\""
Jul 07 03:07:00.870096 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.870072    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"]
Jul 07 03:07:00.973521 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.973479    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-proc\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:00.973705 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.973529    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-podres\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:00.973705 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.973590    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6gcm\" (UniqueName: \"kubernetes.io/projected/696d29f2-36b9-42fa-9557-d53b18ee5172-kube-api-access-n6gcm\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:00.973705 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.973631    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-lib-modules\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:00.973705 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:00.973655    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-sys\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.074863 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.074829    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-lib-modules\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.074863 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.074867    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-sys\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.075104 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.074918    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-proc\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.075104 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.074954    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-podres\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.075104 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.074988    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6gcm\" (UniqueName: \"kubernetes.io/projected/696d29f2-36b9-42fa-9557-d53b18ee5172-kube-api-access-n6gcm\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.075104 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.074996    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-sys\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.075104 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.075049    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-lib-modules\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.075104 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.075082    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-proc\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.075333 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.075140    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/696d29f2-36b9-42fa-9557-d53b18ee5172-podres\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.084538 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.084508    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6gcm\" (UniqueName: \"kubernetes.io/projected/696d29f2-36b9-42fa-9557-d53b18ee5172-kube-api-access-n6gcm\") pod \"perf-node-gather-daemonset-h4gkg\" (UID: \"696d29f2-36b9-42fa-9557-d53b18ee5172\") " pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.173738 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.173641    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.317321 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.317292    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"]
Jul 07 03:07:01.319713 ip-10-0-142-230 kubenswrapper[2585]: W0707 03:07:01.319683    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod696d29f2_36b9_42fa_9557_d53b18ee5172.slice/crio-4c4e474878e03d718d444e1ffed8b4f28f118970f271dd50e41fd7a6267fb16d WatchSource:0}: Error finding container 4c4e474878e03d718d444e1ffed8b4f28f118970f271dd50e41fd7a6267fb16d: Status 404 returned error can't find the container with id 4c4e474878e03d718d444e1ffed8b4f28f118970f271dd50e41fd7a6267fb16d
Jul 07 03:07:01.606317 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.606283    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg" event={"ID":"696d29f2-36b9-42fa-9557-d53b18ee5172","Type":"ContainerStarted","Data":"76637cef6a6cdd73b8d479263aef69a077712db57749d9fe0b7a756d81518475"}
Jul 07 03:07:01.606533 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.606367    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg" event={"ID":"696d29f2-36b9-42fa-9557-d53b18ee5172","Type":"ContainerStarted","Data":"4c4e474878e03d718d444e1ffed8b4f28f118970f271dd50e41fd7a6267fb16d"}
Jul 07 03:07:01.606533 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.606410    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:01.625591 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.625537    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg" podStartSLOduration=1.625520343 podStartE2EDuration="1.625520343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 03:07:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 03:07:01.624492487 +0000 UTC m=+2947.752579179" watchObservedRunningTime="2026-07-07 03:07:01.625520343 +0000 UTC m=+2947.753607034"
Jul 07 03:07:01.932657 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.932623    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xqnxc_ca58ca66-f969-4f25-85ff-b95d5ac9595b/dns/0.log"
Jul 07 03:07:01.955221 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.955192    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xqnxc_ca58ca66-f969-4f25-85ff-b95d5ac9595b/kube-rbac-proxy/0.log"
Jul 07 03:07:01.983797 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:01.983761    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qbffq_255f2d58-2da4-4f7c-8e74-e2b5d583f2a2/dns-node-resolver/0.log"
Jul 07 03:07:02.491344 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:02.491303    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c849fb7f4-f7sxk_3e54dda5-0900-4d35-bda9-b49eef671a8c/registry/0.log"
Jul 07 03:07:02.538858 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:02.538831    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gkgb6_278a6bcd-e480-4cab-9a83-a53245336588/node-ca/0.log"
Jul 07 03:07:03.796035 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:03.796003    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kh65r_5244c428-40dd-4db7-acb9-5723befa306b/serve-healthcheck-canary/0.log"
Jul 07 03:07:04.269573 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:04.269537    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ffpwx_697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d/kube-rbac-proxy/0.log"
Jul 07 03:07:04.292047 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:04.292020    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ffpwx_697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d/exporter/0.log"
Jul 07 03:07:04.316831 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:04.316802    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ffpwx_697bf4aa-e2cd-4fc2-a2e2-a7ca6d9e300d/extractor/0.log"
Jul 07 03:07:06.463089 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:06.463060    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-f8f7445f4-h4545_5dcf4a39-67aa-4538-a7ca-5ed5824ca25d/manager/0.log"
Jul 07 03:07:06.484885 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:06.484856    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-qtbw2_c398fcf6-4565-46f8-9fcc-8575fa10cfb4/manager/0.log"
Jul 07 03:07:07.620049 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:07.620024    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h4pjz/perf-node-gather-daemonset-h4gkg"
Jul 07 03:07:10.896250 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:10.896167    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sm9dm_4cdfd384-4f36-4a27-8a92-a351a530f803/migrator/0.log"
Jul 07 03:07:10.918370 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:10.918340    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sm9dm_4cdfd384-4f36-4a27-8a92-a351a530f803/graceful-termination/0.log"
Jul 07 03:07:12.120612 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.120570    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-24qgz_bdba4aa3-a5bb-49f2-8dab-424b741e708a/kube-multus/0.log"
Jul 07 03:07:12.513100 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.513068    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s6z8n_e0050f4f-59a5-4548-a62d-cb7c61cff8d1/kube-multus-additional-cni-plugins/0.log"
Jul 07 03:07:12.535575 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.535549    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s6z8n_e0050f4f-59a5-4548-a62d-cb7c61cff8d1/egress-router-binary-copy/0.log"
Jul 07 03:07:12.557653 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.557627    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s6z8n_e0050f4f-59a5-4548-a62d-cb7c61cff8d1/cni-plugins/0.log"
Jul 07 03:07:12.580519 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.580493    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s6z8n_e0050f4f-59a5-4548-a62d-cb7c61cff8d1/bond-cni-plugin/0.log"
Jul 07 03:07:12.603807 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.603784    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s6z8n_e0050f4f-59a5-4548-a62d-cb7c61cff8d1/routeoverride-cni/0.log"
Jul 07 03:07:12.627694 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.627668    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s6z8n_e0050f4f-59a5-4548-a62d-cb7c61cff8d1/whereabouts-cni-bincopy/0.log"
Jul 07 03:07:12.649173 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.649148    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s6z8n_e0050f4f-59a5-4548-a62d-cb7c61cff8d1/whereabouts-cni/0.log"
Jul 07 03:07:12.746128 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.746100    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lklr2_c980fc15-667f-435a-8215-11d614462009/network-metrics-daemon/0.log"
Jul 07 03:07:12.768284 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:12.768212    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lklr2_c980fc15-667f-435a-8215-11d614462009/kube-rbac-proxy/0.log"
Jul 07 03:07:13.849476 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.849431    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-controller/0.log"
Jul 07 03:07:13.868380 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.868356    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/1.log"
Jul 07 03:07:13.868535 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.868417    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovn-acl-logging/0.log"
Jul 07 03:07:13.888000 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.887969    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/kube-rbac-proxy-node/0.log"
Jul 07 03:07:13.912844 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.912811    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/kube-rbac-proxy-ovn-metrics/0.log"
Jul 07 03:07:13.935761 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.935737    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/northd/0.log"
Jul 07 03:07:13.959140 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.959115    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/nbdb/0.log"
Jul 07 03:07:13.979840 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:13.979816    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/sbdb/0.log"
Jul 07 03:07:14.026103 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:14.026069    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s4lcd_fae1d72b-9030-4479-a63c-77d5339e0e22/ovnkube-controller/0.log"
Jul 07 03:07:15.250033 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:15.249958    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-bfq5k_8adfce1b-afab-453b-a0ad-97c3e0dbdf6a/network-check-target-container/0.log"
Jul 07 03:07:16.218790 ip-10-0-142-230 kubenswrapper[2585]: I0707 03:07:16.218769    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-v4vlx_ff0a071f-968b-4626-80e8-c5483ff06d14/iptables-alerter/0.log"