Jul 28 12:53:39.254735 ip-10-0-132-6 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 12:53:39.591487 ip-10-0-132-6 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:53:39.591487 ip-10-0-132-6 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 12:53:39.591487 ip-10-0-132-6 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:53:39.591487 ip-10-0-132-6 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 12:53:39.591487 ip-10-0-132-6 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:53:39.592365 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.592275    2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 12:53:39.596657 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596637    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:53:39.596657 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596656    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596662    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596667    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596672    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596675    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596679    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596683    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596686    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596689    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596692    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596695    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596697    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596700    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596703    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596706    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596715    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596717    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596720    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596723    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:53:39.596719 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596726    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596729    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596732    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596735    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596738    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596740    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596743    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596745    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596748    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596751    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596753    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596757    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596759    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596762    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596765    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596767    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596770    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596773    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596775    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596778    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:53:39.597245 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596780    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596782    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596785    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596788    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596790    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596793    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596795    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596797    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596800    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596803    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596805    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596808    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596810    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596812    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596816    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596819    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596821    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596824    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596827    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596829    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:53:39.598118 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596832    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596835    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596837    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596840    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596843    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596845    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596848    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596850    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596853    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596855    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596858    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596862    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596865    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596868    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596871    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596873    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596876    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596878    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596881    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596883    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:53:39.598861 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596886    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596890    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596893    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596895    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596898    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596900    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596905    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596907    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596910    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596913    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596916    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596918    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596921    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596923    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596926    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596929    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596932    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596934    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.596937    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:53:39.599353 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599335    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599348    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599352    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599356    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599361    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599365    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599368    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599371    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599374    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599377    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599381    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599383    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599386    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599389    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599392    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599413    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599419    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599425    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:53:39.599931 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599429    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599432    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599436    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599439    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599442    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599445    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599447    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599450    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599452    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599455    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599457    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599460    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599462    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599464    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599467    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599471    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599474    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599477    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599481    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599483    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:53:39.600376 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599486    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599488    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599491    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599494    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599496    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599499    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599501    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599504    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599507    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599509    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599512    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599514    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599518    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599521    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599523    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599526    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599528    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599531    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599533    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599536    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:53:39.600891 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599539    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599541    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599544    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599547    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599549    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599552    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599555    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599557    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599560    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599563    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599567    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599569    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599572    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599574    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599577    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599580    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599582    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599585    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599587    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:53:39.601417 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599590    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599592    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599595    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599597    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599600    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599603    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599606    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599608    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599611    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599613    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599616    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599618    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599621    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599624    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599627    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599637    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599641    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599643    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599646    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599648    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:53:39.601913 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599651    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.599654    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600086    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600092    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600095    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600098    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600101    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600104    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600107    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600109    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600114    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600117    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600120    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600123    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600126    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600128    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600131    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600134    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600137    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:53:39.602412 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600139    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600142    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600145    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600148    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600150    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600153    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600156    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600158    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600161    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600164    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600167    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600169    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600173    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600176    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600179    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600183    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600185    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600188    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600191    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600194    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:53:39.602910 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600196    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600199    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600201    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600204    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600207    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600210    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600212    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600215    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600217    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600220    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600222    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600225    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600228    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600231    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600234    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600236    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600239    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600242    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600245    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600248    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:53:39.603413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600254    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600257    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600259    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600262    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600264    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600267    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600269    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600272    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600275    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600277    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600280    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600283    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600285    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600288    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600290    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600293    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600295    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600298    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600300    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600303    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:53:39.603902 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600306    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600309    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600311    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600314    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600316    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600319    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600322    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600326    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600329    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600331    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600334    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600337    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600341    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600343    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600347    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600351    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600354    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600357    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600360    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600362    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:53:39.604513 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600365    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.600368    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600458    2573 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600466    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600472    2573 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600477    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600482    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600485    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600490    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600494    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600498    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600501    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600505    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600508    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600511    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600514    2573 flags.go:64] FLAG: --cgroup-root=""
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600517    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600521    2573 flags.go:64] FLAG: --client-ca-file=""
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600524    2573 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600527    2573 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600531    2573 flags.go:64] FLAG: --cluster-domain=""
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600534    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600537    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600541    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 12:53:39.604990 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600545    2573 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600555    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600557    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600561    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600564    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600567    2573 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600570    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600573    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600576    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600579    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600583    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600586    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600589    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600591    2573 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600594    2573 flags.go:64] FLAG: --enable-server="true"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600597    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600602    2573 flags.go:64] FLAG: --event-burst="100"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600605    2573 flags.go:64] FLAG: --event-qps="50"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600608    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600611    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600613    2573 flags.go:64] FLAG: --eviction-hard=""
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600617    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600620    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600623    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600626    2573 flags.go:64] FLAG: --eviction-soft=""
Jul 28 12:53:39.605574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600629    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600632    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600636    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600639    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600641    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600646    2573 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600649    2573 flags.go:64] FLAG: --feature-gates=""
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600653    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600656    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600660    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600663    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600666    2573 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600669    2573 flags.go:64] FLAG: --help="false"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600672    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600675    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600678    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600681    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600684    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600687    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600690    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600693    2573 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600696    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600699    2573 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 12:53:39.606170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600702    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600705    2573 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600708    2573 flags.go:64] FLAG: --kube-reserved=""
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600710    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600713    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600716    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600719    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600722    2573 flags.go:64] FLAG: --lock-file=""
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600725    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600728    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600730    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600736    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600739    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600742    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600749    2573 flags.go:64] FLAG: --logging-format="text"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600752    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600755    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600758    2573 flags.go:64] FLAG: --manifest-url=""
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600761    2573 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600766    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600769    2573 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600773    2573 flags.go:64] FLAG: --max-pods="110"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600776    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600779    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600782    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 12:53:39.606765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600785    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600788    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600791    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600794    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600802    2573 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600805    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600807    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600810    2573 flags.go:64] FLAG: --pod-cidr=""
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600813    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600820    2573 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600822    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600825    2573 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600828    2573 flags.go:64] FLAG: --port="10250"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600831    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600835    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0699b32ba9240047e"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600838    2573 flags.go:64] FLAG: --qos-reserved=""
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600841    2573 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600843    2573 flags.go:64] FLAG: --register-node="true"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600847    2573 flags.go:64] FLAG: --register-with-taints=""
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600851    2573 flags.go:64] FLAG: --registry-burst="10"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600855    2573 flags.go:64] FLAG: --registry-qps="5"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600858    2573 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600861    2573 flags.go:64] FLAG: --reserved-memory=""
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600865    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600868    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 12:53:39.607438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600871    2573 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600874    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600876    2573 flags.go:64] FLAG: --runonce="false"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600879    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600882    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600885    2573 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600888    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600891    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600894    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600898    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600901    2573 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600904    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600907    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600910    2573 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600912    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600915    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600918    2573 flags.go:64] FLAG: --system-cgroups=""
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600921    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600926    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600929    2573 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600932    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600936    2573 flags.go:64] FLAG: --tls-min-version=""
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600938    2573 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600941    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600944    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 12:53:39.608082 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600946    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600949    2573 flags.go:64] FLAG: --v="2"
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600953    2573 flags.go:64] FLAG: --version="false"
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600962    2573 flags.go:64] FLAG: --vmodule=""
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600967    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.600971    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601088    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601093    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601096    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601099    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601102    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601104    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601107    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601110    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601113    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601115    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601118    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601121    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601123    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601125    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601128    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:53:39.608748 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601131    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601134    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601136    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601139    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601143    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601147    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601150    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601152    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601155    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601158    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601161    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601163    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601166    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601168    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601172    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601176    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601179    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601182    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601184    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:53:39.609263 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601187    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601190    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601193    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601195    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601198    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601200    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601203    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601205    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601208    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601211    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601214    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601216    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601219    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601221    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601224    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601226    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601229    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601232    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601234    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601237    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:53:39.609752 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601239    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601241    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601244    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601247    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601249    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601252    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601255    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601258    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601262    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601266    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601268    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601271    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601273    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601276    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601280    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601283    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601286    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601289    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601292    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601295    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:53:39.610273 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601297    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601300    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601303    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601305    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601308    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601311    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601313    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601316    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601318    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601321    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601324    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601326    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601329    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601331    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601335    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601339    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601342    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601345    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601347    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601351    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:53:39.610788 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601356    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601358    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601361    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601364    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.601367    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.601998    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.608828    2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.608844    2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.608861    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.608867    2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608927    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608933    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608938    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:53:39.611275 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608942    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608946    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608949    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608952    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608955    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608957    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608960    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608963    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608966    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608969    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608971    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608974    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608976    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608979    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608982    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608984    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608987    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608990    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608993    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:53:39.611620 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.608995    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609008    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609017    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609021    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609024    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609027    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609030    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609033    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609036    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609040    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609042    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609045    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609048    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609051    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609054    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609056    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609059    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609061    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609064    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609067    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:53:39.612086 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609070    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609072    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609075    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609078    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609080    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609083    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609086    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609088    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609091    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609093    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609096    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609098    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609101    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609104    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609106    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609110    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609112    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609115    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609118    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609121    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:53:39.612631 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609124    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609127    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609129    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609132    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609136    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609139    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609142    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609145    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609147    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609150    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609153    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609155    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609158    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609161    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609163    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609166    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609169    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609171    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609174    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609176    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:53:39.613143 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609179    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609181    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609184    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609186    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609189    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609192    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609195    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609197    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609201    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609203    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609206    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609210    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609213    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609216    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609218    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609221    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:53:39.613821 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609223    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.609228    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609345    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609351    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609355    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609359    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609362    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609365    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609368    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609370    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609373    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609376    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609378    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609381    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609383    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:53:39.614294 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609386    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609389    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609391    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609416    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609420    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609423    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609427    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609429    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609432    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609434    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609438    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609441    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609444    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609447    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609450    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609453    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609456    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609458    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609462    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:53:39.615001 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609464    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609467    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609469    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609472    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609475    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609477    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609480    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609483    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609486    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609488    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609491    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609493    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609495    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609498    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609501    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609503    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609506    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609508    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609511    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609513    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:53:39.615616 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609516    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609520    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609523    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609526    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609530    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609533    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609536    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609538    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609541    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609544    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609546    2573 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609549    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609551    2573 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609554    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609557    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609559    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609562    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609564    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609568    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609570    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:53:39.616306 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609573    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609575    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609578    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609580    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609583    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609585    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609588    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609590    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609593    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609595    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609598    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609600    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609603    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609605    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609608    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609611    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609613    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609617    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609619    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609622    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:53:39.616933 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609625    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609627    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609631    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609634    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609637    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609640    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:39.609643    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.609648    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.609770    2573 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.611917    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.613214    2573 server.go:1019] "Starting client certificate rotation"
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.613326    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.613379    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 12:53:39.617453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.615358    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 12:53:39.617782 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.616572    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 12:53:39.630889 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.630863    2573 log.go:25] "Validated CRI v1 runtime API"
Jul 28 12:53:39.633206 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.633192    2573 log.go:25] "Validated CRI v1 image API"
Jul 28 12:53:39.634056 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.634039    2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 12:53:39.638615 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.638595    2573 fs.go:135] Filesystem UUIDs: map[212a762d-b88a-4a2b-95f1-578fa275f755:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eb2faeed-c40f-4bde-9b76-6151a00a392f:/dev/nvme0n1p3]
Jul 28 12:53:39.638664 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.638615    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 12:53:39.642740 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.642715    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 12:53:39.644392 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.644276    2573 manager.go:216] Machine: {Timestamp:2026-07-28 12:53:39.642361368 +0000 UTC m=+0.287666657 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102038 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2072b47df416cf14f1e8fa24a7bd2c SystemUUID:ec2072b4-7df4-16cf-14f1-e8fa24a7bd2c BootID:568564fb-2ce5-409a-8b9b-5e2ad453c9c7 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7d:fa:cb:3c:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7d:fa:cb:3c:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:da:b7:68:3f:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 12:53:39.644392 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.644387    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 12:53:39.644553 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.644539    2573 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 12:53:39.647600 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.647572    2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 12:53:39.647744 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.647602    2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 12:53:39.648295 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.648285    2573 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 12:53:39.648328 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.648296    2573 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 12:53:39.648328 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.648303    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 12:53:39.648776 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.648766    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 12:53:39.649826 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.649816    2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 12:53:39.651685 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.651673    2573 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 12:53:39.651808 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.651798    2573 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 12:53:39.651844 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.651833    2573 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 12:53:39.651876 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.651849    2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 12:53:39.651876 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.651864    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 12:53:39.651876 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.651872    2573 kubelet.go:398] "Adding apiserver pod source"
Jul 28 12:53:39.651987 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.651882    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 12:53:39.652211 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.652197    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 12:53:39.652275 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.652214    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 12:53:39.654261 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.654236    2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 12:53:39.656364 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.656346    2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 12:53:39.656471 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.656386    2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 12:53:39.658141 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658127    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658145    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658154    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658161    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658174    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658182    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658194    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658202    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658212    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 12:53:39.658222 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658221    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 12:53:39.658491 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658233    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 12:53:39.658491 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.658242    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 12:53:39.659557 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.659547    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 12:53:39.659607 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.659560    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 12:53:39.661762 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.661729    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 12:53:39.661762 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.661742    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 12:53:39.661887 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.661832    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 12:53:39.662800 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.662789    2573 server.go:1268] "Started kubelet"
Jul 28 12:53:39.662897 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.662868    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 12:53:39.662950 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.662890    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 12:53:39.663009 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.662993    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 12:53:39.666472 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.666449    2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 12:53:39.666733 ip-10-0-132-6 systemd[1]: Started Kubernetes Kubelet.
Jul 28 12:53:39.666982 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.666807    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v8pdt"
Jul 28 12:53:39.668379 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.668369    2573 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 12:53:39.670229 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.670206    2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 12:53:39.673843 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.673827    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v8pdt"
Jul 28 12:53:39.674258 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.672764    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-6.ec2.internal.18c6753fe9ac6ea3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-6.ec2.internal,UID:ip-10-0-132-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-6.ec2.internal,},FirstTimestamp:2026-07-28 12:53:39.662761635 +0000 UTC m=+0.308066923,LastTimestamp:2026-07-28 12:53:39.662761635 +0000 UTC m=+0.308066923,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-6.ec2.internal,}"
Jul 28 12:53:39.678427 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.678405    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 12:53:39.678520 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.678389    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 12:53:39.678520 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.678485    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:39.679112 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.679091    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 12:53:39.679112 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.679106    2573 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 12:53:39.679241 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.679129    2573 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 12:53:39.679390 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.679373    2573 factory.go:55] Registering systemd factory
Jul 28 12:53:39.679390 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.679411    2573 factory.go:223] Registration of the systemd container factory successfully
Jul 28 12:53:39.680062 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.680002    2573 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 12:53:39.680062 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.680012    2573 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 12:53:39.680519 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.680504    2573 factory.go:153] Registering CRI-O factory
Jul 28 12:53:39.680612 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.680522    2573 factory.go:223] Registration of the crio container factory successfully
Jul 28 12:53:39.680612 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.680574    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 12:53:39.680612 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.680597    2573 factory.go:103] Registering Raw factory
Jul 28 12:53:39.680755 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.680616    2573 manager.go:1196] Started watching for new ooms in manager
Jul 28 12:53:39.681224 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.681029    2573 manager.go:349] Starting recovery of all containers
Jul 28 12:53:39.682787 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.682758    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:53:39.686877 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.686841    2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-6.ec2.internal\" not found" node="ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.691238 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.691151    2573 manager.go:354] Recovery completed
Jul 28 12:53:39.695079 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.695067    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:53:39.697069 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.697053    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:53:39.697132 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.697083    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:53:39.697132 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.697095    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:53:39.697629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.697613    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 12:53:39.697678 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.697629    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 12:53:39.697678 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.697651    2573 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 12:53:39.697678 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.697620    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:53:39.698322 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.698308    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:53:39.698366 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.698333    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:53:39.698366 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.698343    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:53:39.699279 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.699267    2573 policy_none.go:49] "None policy: Start"
Jul 28 12:53:39.699324 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.699283    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 12:53:39.699324 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.699294    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 12:53:39.700660 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.700646    2573 policy_none.go:47] "Start"
Jul 28 12:53:39.741165 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.741000    2573 manager.go:337] "Starting Device Plugin manager"
Jul 28 12:53:39.741273 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.741181    2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 12:53:39.741273 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.741191    2573 server.go:87] "Starting device plugin registration server"
Jul 28 12:53:39.741441 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.741430    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 12:53:39.741494 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.741442    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 12:53:39.741578 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.741556    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 12:53:39.741687 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.741659    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 12:53:39.741687 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.741669    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 12:53:39.742173 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.742145    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 12:53:39.742234 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.742194    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:39.841806 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.841721    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:53:39.842767 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.842739    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:53:39.842832 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.842774    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:53:39.842832 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.842788    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:53:39.842832 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.842816    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.851223 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.851204    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.851329 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.851229    2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-6.ec2.internal\": node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:39.855911 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.855883    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 12:53:39.857103 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.857086    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 12:53:39.857190 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.857127    2573 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 12:53:39.857190 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.857152    2573 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 12:53:39.857284 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.857194    2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 12:53:39.858991 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.858967    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:53:39.866030 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.866013    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:39.957997 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.957957    2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"]
Jul 28 12:53:39.958144 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.958059    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:53:39.959132 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.959115    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:53:39.959235 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.959145    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:53:39.959235 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.959155    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:53:39.960366 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.960354    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:53:39.960504 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.960477    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.960552 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.960524    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:53:39.961155 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.961137    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:53:39.961272 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.961168    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:53:39.961272 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.961183    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:53:39.961272 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.961196    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:53:39.961272 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.961221    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:53:39.961272 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.961235    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:53:39.962550 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.962533    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.962637 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.962558    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:53:39.963235 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.963219    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:53:39.963300 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.963247    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:53:39.963300 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.963257    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:53:39.966152 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.966137    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:39.982348 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.982321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/536c5ff9fffd6975f95e3ba4d3456e6e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-6.ec2.internal\" (UID: \"536c5ff9fffd6975f95e3ba4d3456e6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.982487 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.982351    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/829405aac64a32fa494fe48895b7d56c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"829405aac64a32fa494fe48895b7d56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.982487 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:39.982377    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/829405aac64a32fa494fe48895b7d56c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"829405aac64a32fa494fe48895b7d56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:39.996241 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:39.996214    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-6.ec2.internal\" not found" node="ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.000772 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.000755    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-6.ec2.internal\" not found" node="ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.066623 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.066591    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.083181 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.083152    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/536c5ff9fffd6975f95e3ba4d3456e6e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-6.ec2.internal\" (UID: \"536c5ff9fffd6975f95e3ba4d3456e6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.083269 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.083188    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/536c5ff9fffd6975f95e3ba4d3456e6e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-6.ec2.internal\" (UID: \"536c5ff9fffd6975f95e3ba4d3456e6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.083269 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.083197    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/829405aac64a32fa494fe48895b7d56c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"829405aac64a32fa494fe48895b7d56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.083269 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.083226    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/829405aac64a32fa494fe48895b7d56c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"829405aac64a32fa494fe48895b7d56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.083269 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.083249    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/829405aac64a32fa494fe48895b7d56c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"829405aac64a32fa494fe48895b7d56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.083387 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.083294    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/829405aac64a32fa494fe48895b7d56c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"829405aac64a32fa494fe48895b7d56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.167485 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.167450    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.274598 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.274559    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.299965 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.299937    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.305551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.305535    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.376035 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.375933    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.479317 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.479293    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.485323 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.485298    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:53:40.580210 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.580176    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.613106 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.613077    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 12:53:40.613584 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.613252    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 12:53:40.613584 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.613292    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 12:53:40.613584 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.613307    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 12:53:40.678739 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.678642    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 12:48:40 +0000 UTC" deadline="2027-12-27 14:13:31.617894154 +0000 UTC"
Jul 28 12:53:40.678739 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.678678    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12409h19m50.939220884s"
Jul 28 12:53:40.680345 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.680326    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.781330 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.781295    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.881874 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:40.881852    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found"
Jul 28 12:53:40.891290 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:40.891256    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829405aac64a32fa494fe48895b7d56c.slice/crio-12ab5aceec7d4ef55bb0f1397622b2559229e9a59088c5293d9d5759d35278ca WatchSource:0}: Error finding container 12ab5aceec7d4ef55bb0f1397622b2559229e9a59088c5293d9d5759d35278ca: Status 404 returned error can't find the container with id 12ab5aceec7d4ef55bb0f1397622b2559229e9a59088c5293d9d5759d35278ca
Jul 28 12:53:40.891519 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:40.891504    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536c5ff9fffd6975f95e3ba4d3456e6e.slice/crio-c6cd076aa2744537bd8a5ff116f2e6abdd28901af7d4cf9b056457ebe2de37ad WatchSource:0}: Error finding container c6cd076aa2744537bd8a5ff116f2e6abdd28901af7d4cf9b056457ebe2de37ad: Status 404 returned error can't find the container with id c6cd076aa2744537bd8a5ff116f2e6abdd28901af7d4cf9b056457ebe2de37ad
Jul 28 12:53:40.898029 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.898014    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:53:40.963660 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.963576    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:53:40.979465 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.979441    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:40.996029 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.996004    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 12:53:40.997101 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:40.997085    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"
Jul 28 12:53:41.004750 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.004727    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 12:53:41.655608 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.655571    2573 apiserver.go:52] "Watching apiserver"
Jul 28 12:53:41.668311 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.668088    2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 12:53:41.669326 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.669300    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qv6xh","openshift-dns/node-resolver-sr76w","openshift-image-registry/node-ca-bzkx5","openshift-multus/multus-additional-cni-plugins-fdhtq","openshift-network-diagnostics/network-check-target-9qcmh","kube-system/global-pull-secret-syncer-6vk6b","kube-system/konnectivity-agent-s5s82","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal","openshift-multus/multus-h27hs","openshift-multus/network-metrics-daemon-r8nsf","openshift-network-operator/iptables-alerter-vcl8j","openshift-ovn-kubernetes/ovnkube-node-vr6js"]
Jul 28 12:53:41.671249 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.671221    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.671356 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.671310    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:41.673009 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.672900    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.674044 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.674021    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.675355 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.675334    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.676830 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.676808    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4dnwb\""
Jul 28 12:53:41.676992 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.676969    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 12:53:41.677093 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.677071    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 12:53:41.677317 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.677289    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:53:41.677317 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.677310    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.677496 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.677351    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 12:53:41.677496 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.677440    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 12:53:41.677751 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.677725    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rsznk\""
Jul 28 12:53:41.677984 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.677941    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 12:53:41.678280 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.678159    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 12:53:41.678586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.678568    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dqrr6\""
Jul 28 12:53:41.678773 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.678756    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 12:53:41.678862 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.678845    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 12:53:41.679634 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.679447    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:41.679634 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.679519    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:41.680739 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.680530    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 12:53:41.680837 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.680801    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 12:53:41.680896 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.680842    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 12:53:41.680896 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.680855    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 12:53:41.680896 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.680857    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 12:53:41.681042 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.681022    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5mf46\""
Jul 28 12:53:41.681124 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.681098    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.683263 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.682481    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.684012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.683991    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 12:53:41.684167 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.684013    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 12:53:41.684167 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.684040    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 12:53:41.684167 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.683994    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dmsjd\""
Jul 28 12:53:41.684848 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.684827    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.685333 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.685309    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 12:53:41.685768 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.685643    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 12:53:41.685768 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.685648    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 12:53:41.685927 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.685910    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wt2rq\""
Jul 28 12:53:41.686528 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.686117    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:41.686845 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.686658    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 12:53:41.686845 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.686810    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 12:53:41.687081 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.687064    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 12:53:41.687692 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.687667    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:53:41.688005 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.687981    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r4mt6\""
Jul 28 12:53:41.688085 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.687980    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 12:53:41.688307 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.688286    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.689908 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.689886    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:41.690007 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.689971    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:41.690397 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.690379    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mm6mg\""
Jul 28 12:53:41.690496 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.690404    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 12:53:41.690496 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.690450    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 12:53:41.690974 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.690951    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh5nf\" (UniqueName: \"kubernetes.io/projected/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-kube-api-access-rh5nf\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.691041 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.690989    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b89a2f2a-171a-40d2-b23f-1523973a6329-host\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.691041 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691017    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcbh8\" (UniqueName: \"kubernetes.io/projected/b89a2f2a-171a-40d2-b23f-1523973a6329-kube-api-access-lcbh8\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.691151 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691048    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.691151 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691112    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:41.691151 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691137    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-registration-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.691312 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691162    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8h28\" (UniqueName: \"kubernetes.io/projected/d652d4b5-709d-42ce-89ee-6d05d54608f6-kube-api-access-w8h28\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.691312 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691192    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.691312 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691220    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.691312 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691258    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t8wr\" (UniqueName: \"kubernetes.io/projected/bea7b828-fe51-446f-a271-8900e62c55a3-kube-api-access-8t8wr\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.691312 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691290    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691314    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-host-slash\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691352    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/282de852-6184-4004-a1fd-689bd1949d36-dbus\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691375    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691406    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-cnibin\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691429    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b89a2f2a-171a-40d2-b23f-1523973a6329-serviceca\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691480    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/282de852-6184-4004-a1fd-689bd1949d36-kubelet-config\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691509    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71951990-b81f-4883-94f4-cd91e45251d2-hosts-file\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691533    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.691586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691559    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-sys-fs\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691628    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcc9f\" (UniqueName: \"kubernetes.io/projected/71951990-b81f-4883-94f4-cd91e45251d2-kube-api-access-hcc9f\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691652    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-system-cni-dir\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691681    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-device-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691704    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-iptables-alerter-script\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691729    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71951990-b81f-4883-94f4-cd91e45251d2-tmp-dir\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691753    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691778    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-socket-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.692034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.691802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-os-release\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.692385 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.692360    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-trbw7\""
Jul 28 12:53:41.694175 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.694147    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 12:53:41.696724 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.696699    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 12:53:41.699327 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.699253    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:53:41.712030 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.712005    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qpnh4"
Jul 28 12:53:41.718019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.717996    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qpnh4"
Jul 28 12:53:41.780669 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.780591    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 12:53:41.793009 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.792975    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-etc-kubernetes\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.793178 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793020    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-log-socket\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.793178 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793049    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-lib-modules\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.793178 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793081    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.793178 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793112    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t8wr\" (UniqueName: \"kubernetes.io/projected/bea7b828-fe51-446f-a271-8900e62c55a3-kube-api-access-8t8wr\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.793178 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793142    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793167    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-host-slash\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793227    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-cni-bin\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793256    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-cnibin\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793323    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b89a2f2a-171a-40d2-b23f-1523973a6329-serviceca\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793347    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-cnibin\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793372    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-conf-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793397    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-modprobe-d\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793420    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-host\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.793460 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793461    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71951990-b81f-4883-94f4-cd91e45251d2-hosts-file\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793491    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793548    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-k8s-cni-cncf-io\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793582    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7bwf\" (UniqueName: \"kubernetes.io/projected/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-kube-api-access-b7bwf\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793610    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793635    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-sys\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793659    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/323970a5-f292-4c73-82c5-74b4cf765494-agent-certs\") pod \"konnectivity-agent-s5s82\" (UID: \"323970a5-f292-4c73-82c5-74b4cf765494\") " pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793705    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcc9f\" (UniqueName: \"kubernetes.io/projected/71951990-b81f-4883-94f4-cd91e45251d2-kube-api-access-hcc9f\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793733    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-system-cni-dir\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793760    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-device-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793781    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793806    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-iptables-alerter-script\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793869    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-run-ovn-kubernetes\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793894    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-env-overrides\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.793921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793921    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysctl-d\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.793954    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794000    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b89a2f2a-171a-40d2-b23f-1523973a6329-serviceca\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-hostroot\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794005    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-host-slash\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794049    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-cnibin\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794067    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71951990-b81f-4883-94f4-cd91e45251d2-hosts-file\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794068    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-cni-netd\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794096    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-device-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794137    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-run\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794166    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794209    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-kubelet\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794235    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-daemon-config\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794259    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-slash\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794284    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7nsj\" (UniqueName: \"kubernetes.io/projected/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-kube-api-access-b7nsj\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794327    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh5nf\" (UniqueName: \"kubernetes.io/projected/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-kube-api-access-rh5nf\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794336    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-system-cni-dir\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.794625 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794376    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-run-netns\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794402    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6cgq\" (UniqueName: \"kubernetes.io/projected/49c7f34f-5a6c-4986-93e4-16ff1577d508-kube-api-access-w6cgq\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794432    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794448    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-registration-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794465    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8h28\" (UniqueName: \"kubernetes.io/projected/d652d4b5-709d-42ce-89ee-6d05d54608f6-kube-api-access-w8h28\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794551    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-netns\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794570    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-cni-multus\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794587    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysconfig\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794606    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794625    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-socket-dir-parent\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794631    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794646    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-multus-certs\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794650    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-iptables-alerter-script\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794684    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f469l\" (UniqueName: \"kubernetes.io/projected/8d74a432-2d04-4ccb-bd11-2ff510789087-kube-api-access-f469l\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794702    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-kubernetes\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794725    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/282de852-6184-4004-a1fd-689bd1949d36-dbus\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.795413 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794737    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-registration-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794750    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794808    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.794939    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794970    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-cni-binary-copy\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.794981    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/282de852-6184-4004-a1fd-689bd1949d36-dbus\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.795048    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret podName:282de852-6184-4004-a1fd-689bd1949d36 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:42.295002844 +0000 UTC m=+2.940308125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret") pod "global-pull-secret-syncer-6vk6b" (UID: "282de852-6184-4004-a1fd-689bd1949d36") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-cni-bin\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795102    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-var-lib-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795134    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/282de852-6184-4004-a1fd-689bd1949d36-kubelet-config\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795163    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-sys-fs\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795177    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/282de852-6184-4004-a1fd-689bd1949d36-kubelet-config\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795187    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-etc-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795217    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-sys-fs\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795283    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-ovn\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795319    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d74a432-2d04-4ccb-bd11-2ff510789087-ovn-node-metrics-cert\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.796012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795365    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-systemd-units\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795389    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-node-log\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795418    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysctl-conf\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795454    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-systemd\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795509    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71951990-b81f-4883-94f4-cd91e45251d2-tmp-dir\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795575    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795643    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795659    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-socket-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795690    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-var-lib-kubelet\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795719    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/323970a5-f292-4c73-82c5-74b4cf765494-konnectivity-ca\") pod \"konnectivity-agent-s5s82\" (UID: \"323970a5-f292-4c73-82c5-74b4cf765494\") " pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795748    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-os-release\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795761    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d652d4b5-709d-42ce-89ee-6d05d54608f6-socket-dir\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795773    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-cni-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795791    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-tuned\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-tmp\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795843    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bea7b828-fe51-446f-a271-8900e62c55a3-os-release\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795853    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71951990-b81f-4883-94f4-cd91e45251d2-tmp-dir\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.796803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795848    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b89a2f2a-171a-40d2-b23f-1523973a6329-host\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795888    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b89a2f2a-171a-40d2-b23f-1523973a6329-host\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795891    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcbh8\" (UniqueName: \"kubernetes.io/projected/b89a2f2a-171a-40d2-b23f-1523973a6329-kube-api-access-lcbh8\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795921    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-system-cni-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795947    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-kubelet\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795972    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-systemd\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.795996    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.796021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-ovnkube-config\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.796045    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-ovnkube-script-lib\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.796092    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.796126    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-os-release\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.797651 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.796616    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bea7b828-fe51-446f-a271-8900e62c55a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.802381 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.802360    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 12:53:41.802990 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.802870    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:53:41.802990 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.802895    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:53:41.802990 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.802910    2573 projected.go:196] Error preparing data for projected volume kube-api-access-n2cqz for pod openshift-network-diagnostics/network-check-target-9qcmh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:41.802990 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.802978    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz podName:3499e17e-83e5-40a6-9e3e-b1684607cfe1 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:42.302949707 +0000 UTC m=+2.948255004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n2cqz" (UniqueName: "kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz") pod "network-check-target-9qcmh" (UID: "3499e17e-83e5-40a6-9e3e-b1684607cfe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:41.807123 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.807067    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcc9f\" (UniqueName: \"kubernetes.io/projected/71951990-b81f-4883-94f4-cd91e45251d2-kube-api-access-hcc9f\") pod \"node-resolver-sr76w\" (UID: \"71951990-b81f-4883-94f4-cd91e45251d2\") " pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:41.807123 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.807072    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t8wr\" (UniqueName: \"kubernetes.io/projected/bea7b828-fe51-446f-a271-8900e62c55a3-kube-api-access-8t8wr\") pod \"multus-additional-cni-plugins-fdhtq\" (UID: \"bea7b828-fe51-446f-a271-8900e62c55a3\") " pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:41.807402 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.807376    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh5nf\" (UniqueName: \"kubernetes.io/projected/d3d01e36-e904-4ef1-bd3e-e2ac5b99660a-kube-api-access-rh5nf\") pod \"iptables-alerter-vcl8j\" (UID: \"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a\") " pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.807902 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.807874    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8h28\" (UniqueName: \"kubernetes.io/projected/d652d4b5-709d-42ce-89ee-6d05d54608f6-kube-api-access-w8h28\") pod \"aws-ebs-csi-driver-node-pwc7v\" (UID: \"d652d4b5-709d-42ce-89ee-6d05d54608f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:41.813859 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.813828    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcbh8\" (UniqueName: \"kubernetes.io/projected/b89a2f2a-171a-40d2-b23f-1523973a6329-kube-api-access-lcbh8\") pod \"node-ca-bzkx5\" (UID: \"b89a2f2a-171a-40d2-b23f-1523973a6329\") " pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:41.862731 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.862668    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" event={"ID":"829405aac64a32fa494fe48895b7d56c","Type":"ContainerStarted","Data":"12ab5aceec7d4ef55bb0f1397622b2559229e9a59088c5293d9d5759d35278ca"}
Jul 28 12:53:41.863984 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.863953    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" event={"ID":"536c5ff9fffd6975f95e3ba4d3456e6e","Type":"ContainerStarted","Data":"c6cd076aa2744537bd8a5ff116f2e6abdd28901af7d4cf9b056457ebe2de37ad"}
Jul 28 12:53:41.897226 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897190    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-systemd\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.897226 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897239    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-var-lib-kubelet\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897266    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/323970a5-f292-4c73-82c5-74b4cf765494-konnectivity-ca\") pod \"konnectivity-agent-s5s82\" (UID: \"323970a5-f292-4c73-82c5-74b4cf765494\") " pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897294    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-cni-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897341    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-systemd\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897351    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-tuned\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897382    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-cni-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897402    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-tmp\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-var-lib-kubelet\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.897551 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897540    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-system-cni-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897602    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-system-cni-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897692    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-kubelet\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897722    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-systemd\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897749    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897775    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-ovnkube-config\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897823    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-ovnkube-script-lib\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897835    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-kubelet\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897857    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-os-release\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897877    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-systemd\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-etc-kubernetes\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897924    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-log-socket\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.897948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897947    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-lib-modules\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897960    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-os-release\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.897977    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-cni-bin\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.898006    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-cnibin\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.898028    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-conf-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.898048    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-etc-kubernetes\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.898050    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-modprobe-d\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.898102    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/323970a5-f292-4c73-82c5-74b4cf765494-konnectivity-ca\") pod \"konnectivity-agent-s5s82\" (UID: \"323970a5-f292-4c73-82c5-74b4cf765494\") " pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:41.898629 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.898122    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-host\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.900423 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900383    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.900775 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900752    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-ovnkube-script-lib\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.900864 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900841    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-modprobe-d\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.900922 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900884    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-k8s-cni-cncf-io\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.900972 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900917    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7bwf\" (UniqueName: \"kubernetes.io/projected/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-kube-api-access-b7bwf\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.900972 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900957    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901068 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900996    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-sys\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901068 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901030    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/323970a5-f292-4c73-82c5-74b4cf765494-agent-certs\") pod \"konnectivity-agent-s5s82\" (UID: \"323970a5-f292-4c73-82c5-74b4cf765494\") " pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:41.901182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901072    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-run-ovn-kubernetes\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901102    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-env-overrides\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901123    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-tuned\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901136    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysctl-d\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901170    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-hostroot\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901204    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-cni-netd\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901236    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-run\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901268    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901285    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-tmp\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901302    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-kubelet\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901336    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-daemon-config\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901371    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-slash\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901373    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-run-ovn-kubernetes\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901412    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7nsj\" (UniqueName: \"kubernetes.io/projected/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-kube-api-access-b7nsj\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901440    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-run-netns\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901488    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6cgq\" (UniqueName: \"kubernetes.io/projected/49c7f34f-5a6c-4986-93e4-16ff1577d508-kube-api-access-w6cgq\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901541    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-netns\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901576    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-cni-multus\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901609    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysconfig\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901641    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-socket-dir-parent\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901675    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-multus-certs\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901704    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f469l\" (UniqueName: \"kubernetes.io/projected/8d74a432-2d04-4ccb-bd11-2ff510789087-kube-api-access-f469l\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901719    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-sys\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901738    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-kubernetes\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901792    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-cni-binary-copy\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901819    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-cni-bin\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901857    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-var-lib-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901891    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-etc-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901917    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-ovn\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901935    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d74a432-2d04-4ccb-bd11-2ff510789087-ovn-node-metrics-cert\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.901958 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901957    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-systemd-units\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901979    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-node-log\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.901999    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysctl-conf\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.902140    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysctl-conf\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.902421    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-env-overrides\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.902558    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysctl-d\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.902596    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-hostroot\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.902643    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-cni-netd\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.902738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.902683    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-run\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.903112 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.902760    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:41.903112 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:41.902814    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs podName:49c7f34f-5a6c-4986-93e4-16ff1577d508 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:42.402799621 +0000 UTC m=+3.048104898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs") pod "network-metrics-daemon-r8nsf" (UID: "49c7f34f-5a6c-4986-93e4-16ff1577d508") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:41.903112 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.903012    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-kubelet\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.903473 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.903450    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-daemon-config\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.903596 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.903527    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-slash\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.903800 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.903780    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-run-netns\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.904080 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.904063    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-netns\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.904126 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.904113    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-cni-multus\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.904172 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.904161    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-sysconfig\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.904210 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.904200    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-socket-dir-parent\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.904255 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.904231    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.904300 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.904288    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-multus-certs\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.904645 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.904531    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-etc-kubernetes\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.905123 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905050    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-cni-binary-copy\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.905123 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905077    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-run-ovn\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.905268 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905131    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-var-lib-cni-bin\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.905268 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905209    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-var-lib-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.905268 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905261    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-etc-openvswitch\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.905432 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905325    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-node-log\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.905432 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905359    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-systemd-units\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.905581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905442    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-log-socket\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.905685 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905605    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-lib-modules\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.905685 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905646    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d74a432-2d04-4ccb-bd11-2ff510789087-host-cni-bin\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.905803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905690    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-host\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.905803 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905793    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-cnibin\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.905907 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905867    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-multus-conf-dir\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.905986 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.905943    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-host-run-k8s-cni-cncf-io\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.907091 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.900340    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d74a432-2d04-4ccb-bd11-2ff510789087-ovnkube-config\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.908185 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.908161    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d74a432-2d04-4ccb-bd11-2ff510789087-ovn-node-metrics-cert\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.910704 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.910681    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/323970a5-f292-4c73-82c5-74b4cf765494-agent-certs\") pod \"konnectivity-agent-s5s82\" (UID: \"323970a5-f292-4c73-82c5-74b4cf765494\") " pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:41.912172 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.912145    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7bwf\" (UniqueName: \"kubernetes.io/projected/2b1be2bf-c833-4639-9f5e-f7621efdcb9d-kube-api-access-b7bwf\") pod \"multus-h27hs\" (UID: \"2b1be2bf-c833-4639-9f5e-f7621efdcb9d\") " pod="openshift-multus/multus-h27hs"
Jul 28 12:53:41.913783 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.913579    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6cgq\" (UniqueName: \"kubernetes.io/projected/49c7f34f-5a6c-4986-93e4-16ff1577d508-kube-api-access-w6cgq\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:41.914019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.913998    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f469l\" (UniqueName: \"kubernetes.io/projected/8d74a432-2d04-4ccb-bd11-2ff510789087-kube-api-access-f469l\") pod \"ovnkube-node-vr6js\" (UID: \"8d74a432-2d04-4ccb-bd11-2ff510789087\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:41.914174 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.914151    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7nsj\" (UniqueName: \"kubernetes.io/projected/0e54284b-26d3-42a4-8b8a-a6c1c6c202d8-kube-api-access-b7nsj\") pod \"tuned-qv6xh\" (UID: \"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:41.950782 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.950750    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:53:41.989708 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.989664    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vcl8j"
Jul 28 12:53:41.999465 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:41.999435    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sr76w"
Jul 28 12:53:42.008919 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.008893    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bzkx5"
Jul 28 12:53:42.015913 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.015885    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fdhtq"
Jul 28 12:53:42.024969 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.024939    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v"
Jul 28 12:53:42.036066 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.036042    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:53:42.042153 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.042132    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qv6xh"
Jul 28 12:53:42.049364 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.049340    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:53:42.055323 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.055301    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h27hs"
Jul 28 12:53:42.305381 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.305298    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:42.305381 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.305337    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:42.305574 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.305446    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:42.305574 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.305463    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:53:42.305574 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.305488    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:53:42.305574 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.305498    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret podName:282de852-6184-4004-a1fd-689bd1949d36 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:43.305483049 +0000 UTC m=+3.950788326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret") pod "global-pull-secret-syncer-6vk6b" (UID: "282de852-6184-4004-a1fd-689bd1949d36") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:42.305574 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.305500    2573 projected.go:196] Error preparing data for projected volume kube-api-access-n2cqz for pod openshift-network-diagnostics/network-check-target-9qcmh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:42.305574 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.305545    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz podName:3499e17e-83e5-40a6-9e3e-b1684607cfe1 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:43.30553199 +0000 UTC m=+3.950837268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2cqz" (UniqueName: "kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz") pod "network-check-target-9qcmh" (UID: "3499e17e-83e5-40a6-9e3e-b1684607cfe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:42.406660 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.406563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:42.406869 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.406747    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:42.406869 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.406829    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs podName:49c7f34f-5a6c-4986-93e4-16ff1577d508 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:43.406807081 +0000 UTC m=+4.052112359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs") pod "network-metrics-daemon-r8nsf" (UID: "49c7f34f-5a6c-4986-93e4-16ff1577d508") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:42.720907 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.720815    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:48:42 +0000 UTC" deadline="2028-03-06 11:44:52.799035518 +0000 UTC"
Jul 28 12:53:42.720907 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.720848    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14086h51m10.078190553s"
Jul 28 12:53:42.847470 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.847430    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d74a432_2d04_4ccb_bd11_2ff510789087.slice/crio-cdab3482fcd51e27272852e71785a3893b8f32c8aa1cd4363606a9b40e1a2a29 WatchSource:0}: Error finding container cdab3482fcd51e27272852e71785a3893b8f32c8aa1cd4363606a9b40e1a2a29: Status 404 returned error can't find the container with id cdab3482fcd51e27272852e71785a3893b8f32c8aa1cd4363606a9b40e1a2a29
Jul 28 12:53:42.848312 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.848268    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea7b828_fe51_446f_a271_8900e62c55a3.slice/crio-bca56b6847a501d7acb620b04f764cd14b83d2d2e38e48fd9996290a37ad45c2 WatchSource:0}: Error finding container bca56b6847a501d7acb620b04f764cd14b83d2d2e38e48fd9996290a37ad45c2: Status 404 returned error can't find the container with id bca56b6847a501d7acb620b04f764cd14b83d2d2e38e48fd9996290a37ad45c2
Jul 28 12:53:42.850382 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.850224    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd652d4b5_709d_42ce_89ee_6d05d54608f6.slice/crio-64dba7ebc99b1a6278bbafa9128a22d40809f51982a89d3ed9b54a3d9a133503 WatchSource:0}: Error finding container 64dba7ebc99b1a6278bbafa9128a22d40809f51982a89d3ed9b54a3d9a133503: Status 404 returned error can't find the container with id 64dba7ebc99b1a6278bbafa9128a22d40809f51982a89d3ed9b54a3d9a133503
Jul 28 12:53:42.850897 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.850869    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e54284b_26d3_42a4_8b8a_a6c1c6c202d8.slice/crio-0fc1a84cc941168bb452025513baed1fb43503be9c0039fdd8f6220fd56a91a3 WatchSource:0}: Error finding container 0fc1a84cc941168bb452025513baed1fb43503be9c0039fdd8f6220fd56a91a3: Status 404 returned error can't find the container with id 0fc1a84cc941168bb452025513baed1fb43503be9c0039fdd8f6220fd56a91a3
Jul 28 12:53:42.851701 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.851535    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d01e36_e904_4ef1_bd3e_e2ac5b99660a.slice/crio-ca2a3ba3d7a45fabdfbe5a62bafa9c718c090fa39a7e53a0c2efaf9701f28a81 WatchSource:0}: Error finding container ca2a3ba3d7a45fabdfbe5a62bafa9c718c090fa39a7e53a0c2efaf9701f28a81: Status 404 returned error can't find the container with id ca2a3ba3d7a45fabdfbe5a62bafa9c718c090fa39a7e53a0c2efaf9701f28a81
Jul 28 12:53:42.855106 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.854956    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1be2bf_c833_4639_9f5e_f7621efdcb9d.slice/crio-45ec892789a7b74567cd592736a7c5033690eb97efd3ccb32717517efda3f1f3 WatchSource:0}: Error finding container 45ec892789a7b74567cd592736a7c5033690eb97efd3ccb32717517efda3f1f3: Status 404 returned error can't find the container with id 45ec892789a7b74567cd592736a7c5033690eb97efd3ccb32717517efda3f1f3
Jul 28 12:53:42.857918 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.857585    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:42.857918 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.857669    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:42.857918 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.857772    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:42.857918 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.857827    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb89a2f2a_171a_40d2_b23f_1523973a6329.slice/crio-8e2cb12e4ea914c1683859c532a265e11b2ff61f4b0efc1049f0608b8a7ecbc9 WatchSource:0}: Error finding container 8e2cb12e4ea914c1683859c532a265e11b2ff61f4b0efc1049f0608b8a7ecbc9: Status 404 returned error can't find the container with id 8e2cb12e4ea914c1683859c532a265e11b2ff61f4b0efc1049f0608b8a7ecbc9
Jul 28 12:53:42.857918 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:42.857893    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:42.861881 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.860415    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323970a5_f292_4c73_82c5_74b4cf765494.slice/crio-a5620b8c02e4fd2e578677b402744e93e71dbee7e7f344651a4aa9302b9b2f14 WatchSource:0}: Error finding container a5620b8c02e4fd2e578677b402744e93e71dbee7e7f344651a4aa9302b9b2f14: Status 404 returned error can't find the container with id a5620b8c02e4fd2e578677b402744e93e71dbee7e7f344651a4aa9302b9b2f14
Jul 28 12:53:42.861881 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:53:42.860861    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71951990_b81f_4883_94f4_cd91e45251d2.slice/crio-219f8369474feb7ed01ab17271971e41b6a0475e8f917e10953f6d9564b99ae0 WatchSource:0}: Error finding container 219f8369474feb7ed01ab17271971e41b6a0475e8f917e10953f6d9564b99ae0: Status 404 returned error can't find the container with id 219f8369474feb7ed01ab17271971e41b6a0475e8f917e10953f6d9564b99ae0
Jul 28 12:53:42.866986 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.866959    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h27hs" event={"ID":"2b1be2bf-c833-4639-9f5e-f7621efdcb9d","Type":"ContainerStarted","Data":"45ec892789a7b74567cd592736a7c5033690eb97efd3ccb32717517efda3f1f3"}
Jul 28 12:53:42.868231 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.868201    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v" event={"ID":"d652d4b5-709d-42ce-89ee-6d05d54608f6","Type":"ContainerStarted","Data":"64dba7ebc99b1a6278bbafa9128a22d40809f51982a89d3ed9b54a3d9a133503"}
Jul 28 12:53:42.869591 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.869556    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sr76w" event={"ID":"71951990-b81f-4883-94f4-cd91e45251d2","Type":"ContainerStarted","Data":"219f8369474feb7ed01ab17271971e41b6a0475e8f917e10953f6d9564b99ae0"}
Jul 28 12:53:42.871162 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.870975    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:53:42.871709 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.871687    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bzkx5" event={"ID":"b89a2f2a-171a-40d2-b23f-1523973a6329","Type":"ContainerStarted","Data":"8e2cb12e4ea914c1683859c532a265e11b2ff61f4b0efc1049f0608b8a7ecbc9"}
Jul 28 12:53:42.873012 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.872988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerStarted","Data":"bca56b6847a501d7acb620b04f764cd14b83d2d2e38e48fd9996290a37ad45c2"}
Jul 28 12:53:42.874149 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.874128    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vcl8j" event={"ID":"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a","Type":"ContainerStarted","Data":"ca2a3ba3d7a45fabdfbe5a62bafa9c718c090fa39a7e53a0c2efaf9701f28a81"}
Jul 28 12:53:42.875321 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.875298    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qv6xh" event={"ID":"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8","Type":"ContainerStarted","Data":"0fc1a84cc941168bb452025513baed1fb43503be9c0039fdd8f6220fd56a91a3"}
Jul 28 12:53:42.876437 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.876393    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"cdab3482fcd51e27272852e71785a3893b8f32c8aa1cd4363606a9b40e1a2a29"}
Jul 28 12:53:42.877702 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:42.877682    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s5s82" event={"ID":"323970a5-f292-4c73-82c5-74b4cf765494","Type":"ContainerStarted","Data":"a5620b8c02e4fd2e578677b402744e93e71dbee7e7f344651a4aa9302b9b2f14"}
Jul 28 12:53:43.316332 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.315954    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:43.316332 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.316244    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:43.316332 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.316150    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:53:43.316332 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.316326    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:53:43.316794 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.316346    2573 projected.go:196] Error preparing data for projected volume kube-api-access-n2cqz for pod openshift-network-diagnostics/network-check-target-9qcmh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:43.316794 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.316370    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:43.316794 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.316434    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz podName:3499e17e-83e5-40a6-9e3e-b1684607cfe1 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:45.316394227 +0000 UTC m=+5.961699529 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2cqz" (UniqueName: "kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz") pod "network-check-target-9qcmh" (UID: "3499e17e-83e5-40a6-9e3e-b1684607cfe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:43.316794 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.316458    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret podName:282de852-6184-4004-a1fd-689bd1949d36 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:45.316445528 +0000 UTC m=+5.961750818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret") pod "global-pull-secret-syncer-6vk6b" (UID: "282de852-6184-4004-a1fd-689bd1949d36") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:43.417048 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.416998    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:43.417235 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.417177    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:43.417316 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.417244    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs podName:49c7f34f-5a6c-4986-93e4-16ff1577d508 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:45.417224118 +0000 UTC m=+6.062529400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs") pod "network-metrics-daemon-r8nsf" (UID: "49c7f34f-5a6c-4986-93e4-16ff1577d508") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:43.721661 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.721495    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:48:42 +0000 UTC" deadline="2028-02-07 07:26:12.251644119 +0000 UTC"
Jul 28 12:53:43.721661 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.721560    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13410h32m28.530088551s"
Jul 28 12:53:43.859797 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.859213    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:43.859797 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:43.859355    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:43.884455 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.884412    2573 generic.go:356] "Generic (PLEG): container finished" podID="829405aac64a32fa494fe48895b7d56c" containerID="74c938b0d39643bc737383c561533440863a0fe0b186f89638e7a0159297f8ca" exitCode=0
Jul 28 12:53:43.884656 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.884533    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" event={"ID":"829405aac64a32fa494fe48895b7d56c","Type":"ContainerDied","Data":"74c938b0d39643bc737383c561533440863a0fe0b186f89638e7a0159297f8ca"}
Jul 28 12:53:43.891921 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:43.890907    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" event={"ID":"536c5ff9fffd6975f95e3ba4d3456e6e","Type":"ContainerStarted","Data":"553faa36a7b69b786e2176b0d6d3eaaed48a950f1dc0de26cfa7a7d88b7f263f"}
Jul 28 12:53:44.857857 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:44.857820    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:44.858383 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:44.857976    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:44.858455 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:44.858393    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:44.858547 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:44.858522    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:44.898578 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:44.898541    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" event={"ID":"829405aac64a32fa494fe48895b7d56c","Type":"ContainerStarted","Data":"3111bfc8e33d4f9b49914dc0f495200ba53b2a604aba4264599111f3a27fa17f"}
Jul 28 12:53:44.917409 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:44.917337    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" podStartSLOduration=3.917319735 podStartE2EDuration="3.917319735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:53:43.925420867 +0000 UTC m=+4.570726170" watchObservedRunningTime="2026-07-28 12:53:44.917319735 +0000 UTC m=+5.562625036"
Jul 28 12:53:45.343857 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:45.343762    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:45.343857 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:45.343827    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:45.344095 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.344017    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:45.344095 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.344090    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret podName:282de852-6184-4004-a1fd-689bd1949d36 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:49.344068327 +0000 UTC m=+9.989373629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret") pod "global-pull-secret-syncer-6vk6b" (UID: "282de852-6184-4004-a1fd-689bd1949d36") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:45.344583 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.344561    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:53:45.344699 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.344592    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:53:45.344699 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.344608    2573 projected.go:196] Error preparing data for projected volume kube-api-access-n2cqz for pod openshift-network-diagnostics/network-check-target-9qcmh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:45.344699 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.344654    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz podName:3499e17e-83e5-40a6-9e3e-b1684607cfe1 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:49.344638532 +0000 UTC m=+9.989943814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2cqz" (UniqueName: "kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz") pod "network-check-target-9qcmh" (UID: "3499e17e-83e5-40a6-9e3e-b1684607cfe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:45.444879 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:45.444833    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:45.445081 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.445056    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:45.445153 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.445131    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs podName:49c7f34f-5a6c-4986-93e4-16ff1577d508 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:49.445108755 +0000 UTC m=+10.090414037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs") pod "network-metrics-daemon-r8nsf" (UID: "49c7f34f-5a6c-4986-93e4-16ff1577d508") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:45.860331 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:45.859697    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:45.860331 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:45.859896    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:46.857441 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:46.857386    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:46.857635 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:46.857537    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:46.858076 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:46.857918    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:46.858076 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:46.858035    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:47.858533 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:47.858497    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:47.858974 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:47.858642    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:48.858351 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:48.858315    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:48.858557 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:48.858471    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:48.858950 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:48.858868    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:48.858950 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:48.858943    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:49.380458 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:49.380416    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:49.380649 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:49.380481    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:49.380649 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.380616    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:49.380760 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.380672    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:53:49.380760 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.380699    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:53:49.380760 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.380715    2573 projected.go:196] Error preparing data for projected volume kube-api-access-n2cqz for pod openshift-network-diagnostics/network-check-target-9qcmh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:49.380760 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.380684    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret podName:282de852-6184-4004-a1fd-689bd1949d36 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:57.380664424 +0000 UTC m=+18.025969706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret") pod "global-pull-secret-syncer-6vk6b" (UID: "282de852-6184-4004-a1fd-689bd1949d36") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:49.380970 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.380777    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz podName:3499e17e-83e5-40a6-9e3e-b1684607cfe1 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:57.38076091 +0000 UTC m=+18.026066202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2cqz" (UniqueName: "kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz") pod "network-check-target-9qcmh" (UID: "3499e17e-83e5-40a6-9e3e-b1684607cfe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:49.481255 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:49.481161    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:49.481455 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.481360    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:49.481455 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.481450    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs podName:49c7f34f-5a6c-4986-93e4-16ff1577d508 nodeName:}" failed. No retries permitted until 2026-07-28 12:53:57.481427385 +0000 UTC m=+18.126732666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs") pod "network-metrics-daemon-r8nsf" (UID: "49c7f34f-5a6c-4986-93e4-16ff1577d508") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:49.861512 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:49.861436    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:49.861954 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:49.861564    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:50.857934 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:50.857901    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:50.858131 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:50.858032    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:50.858131 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:50.858069    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:50.858244 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:50.858191    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:51.861914 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:51.861632    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:51.861914 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:51.861766    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:52.857963 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:52.857883    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:52.858116 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:52.857997    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:52.858116 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:52.858050    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:52.858213 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:52.858155    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:53.861370 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:53.861342    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:53.861774 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:53.861483    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:54.858021 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:54.857984    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:54.858201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:54.858034    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:54.858201 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:54.858117    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:54.858288 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:54.858248    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:55.858065 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:55.858020    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:55.858547 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:55.858160    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:56.857630 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:56.857597    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:56.857859 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:56.857715    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:56.857859 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:56.857764    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:56.857945 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:56.857883    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:57.443839 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:57.443803    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:57.444293 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:57.443885    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:57.444293 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.443960    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:57.444293 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.443986    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:53:57.444293 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.444001    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:53:57.444293 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.444010    2573 projected.go:196] Error preparing data for projected volume kube-api-access-n2cqz for pod openshift-network-diagnostics/network-check-target-9qcmh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:57.444293 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.444023    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret podName:282de852-6184-4004-a1fd-689bd1949d36 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:13.444006513 +0000 UTC m=+34.089311788 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret") pod "global-pull-secret-syncer-6vk6b" (UID: "282de852-6184-4004-a1fd-689bd1949d36") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:53:57.444293 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.444050    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz podName:3499e17e-83e5-40a6-9e3e-b1684607cfe1 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:13.444036797 +0000 UTC m=+34.089342093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2cqz" (UniqueName: "kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz") pod "network-check-target-9qcmh" (UID: "3499e17e-83e5-40a6-9e3e-b1684607cfe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:53:57.544365 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:57.544337    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:57.544551 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.544490    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:57.544551 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.544548    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs podName:49c7f34f-5a6c-4986-93e4-16ff1577d508 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:13.544533505 +0000 UTC m=+34.189838793 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs") pod "network-metrics-daemon-r8nsf" (UID: "49c7f34f-5a6c-4986-93e4-16ff1577d508") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:53:57.857728 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:57.857655    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:57.857880 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:57.857792    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:53:58.857758 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:58.857725    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:53:58.858156 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:58.857852    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:53:58.858156 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:58.857901    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:53:58.858156 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:58.858017    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:53:59.858969 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:53:59.858932    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:53:59.859438 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:53:59.859043    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:00.857526 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:00.857489    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:00.857734 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:00.857539    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:00.857734 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:00.857640    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:54:00.857853 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:00.857769    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:54:01.858739 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.858250    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:01.859563 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:01.858863    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:01.935019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.934988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sr76w" event={"ID":"71951990-b81f-4883-94f4-cd91e45251d2","Type":"ContainerStarted","Data":"172149d0cc628c0f785e8db5a590c47ceef2914f34b827c13fe70c5966f5e1a2"}
Jul 28 12:54:01.936356 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.936322    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bzkx5" event={"ID":"b89a2f2a-171a-40d2-b23f-1523973a6329","Type":"ContainerStarted","Data":"c97e6de23ea65ef90192bf171ba64f8d3d7403b94e863bbf94038fdfaef50110"}
Jul 28 12:54:01.937628 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.937606    2573 generic.go:356] "Generic (PLEG): container finished" podID="bea7b828-fe51-446f-a271-8900e62c55a3" containerID="cda48dc470869a4649be75c10280e95b09c8b933a4624dd37323ca342324a265" exitCode=0
Jul 28 12:54:01.937719 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.937669    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerDied","Data":"cda48dc470869a4649be75c10280e95b09c8b933a4624dd37323ca342324a265"}
Jul 28 12:54:01.939264 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.939244    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qv6xh" event={"ID":"0e54284b-26d3-42a4-8b8a-a6c1c6c202d8","Type":"ContainerStarted","Data":"616e8b365b442f40caea33446ac649c8a10c8cb2198e558ac523dd01842631c3"}
Jul 28 12:54:01.942258 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.942236    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"9bf68dc5458d748287539d6e7c981263b385cef0a64a79b00a45286771efe561"}
Jul 28 12:54:01.942357 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.942325    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"0688d65b1ea585b0a26c3761ae93feaaaf7f7c424a744ef8cb68e3f1b7113d70"}
Jul 28 12:54:01.942456 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.942367    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"64b930d38c0878bafd6b829aa4ca763f0162a61b960426d3597d2913a8154b20"}
Jul 28 12:54:01.942456 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.942428    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"fe1d7205041f4de10ffe7e8467f2d49597b71f77fdee33a96110301bfc8c9137"}
Jul 28 12:54:01.943817 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.943798    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s5s82" event={"ID":"323970a5-f292-4c73-82c5-74b4cf765494","Type":"ContainerStarted","Data":"550e6c448efd66595ae9770a21a63a5174534a2a14db796d85b524c5dc712c1c"}
Jul 28 12:54:01.945273 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.945251    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h27hs" event={"ID":"2b1be2bf-c833-4639-9f5e-f7621efdcb9d","Type":"ContainerStarted","Data":"57d1f47874c21e57ae58f9878f73cac30ca9cd07eb4602450822f6065da40563"}
Jul 28 12:54:01.946659 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.946641    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v" event={"ID":"d652d4b5-709d-42ce-89ee-6d05d54608f6","Type":"ContainerStarted","Data":"f060446db0729d22c9954c18ae612c8f45f8f9278a947a77ebbf371c0210b170"}
Jul 28 12:54:01.952338 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.952302    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sr76w" podStartSLOduration=3.368706888 podStartE2EDuration="21.952272763s" totalImagesPullingTime="18.583565875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:01.952214335 +0000 UTC m=+22.597519632" watchObservedRunningTime="2026-07-28 12:54:01.952272763 +0000 UTC m=+22.597578067"
Jul 28 12:54:01.952752 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.952721    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" podStartSLOduration=20.952712094 podStartE2EDuration="20.952712094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:53:44.918415383 +0000 UTC m=+5.563720688" watchObservedRunningTime="2026-07-28 12:54:01.952712094 +0000 UTC m=+22.598017392"
Jul 28 12:54:01.969749 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.969704    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bzkx5" podStartSLOduration=3.349918211 podStartE2EDuration="21.969691228s" totalImagesPullingTime="18.619773017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:01.96966357 +0000 UTC m=+22.614968868" watchObservedRunningTime="2026-07-28 12:54:01.969691228 +0000 UTC m=+22.614996525"
Jul 28 12:54:01.988807 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:01.988770    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h27hs" podStartSLOduration=3.317722251 podStartE2EDuration="21.9887573s" totalImagesPullingTime="18.671035049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:01.988496031 +0000 UTC m=+22.633801332" watchObservedRunningTime="2026-07-28 12:54:01.9887573 +0000 UTC m=+22.634062597"
Jul 28 12:54:02.036280 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.036240    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qv6xh" podStartSLOduration=3.414722745 podStartE2EDuration="22.036224903s" totalImagesPullingTime="18.621502158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:02.035660915 +0000 UTC m=+22.680966214" watchObservedRunningTime="2026-07-28 12:54:02.036224903 +0000 UTC m=+22.681530197"
Jul 28 12:54:02.857913 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.857881    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:02.857913 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.857898    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:02.858136 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:02.857994    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:54:02.858136 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:02.858099    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:54:02.953183 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.953143    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"b14d66207d96076cc8c3509ae88a80d4e1efff0ed791e94caaa8692f02fa2ca6"}
Jul 28 12:54:02.953708 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.953246    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"f2675376431083c0037e450cb132233ddf968abbc4ec69f7bb5da05e2aa37e1f"}
Jul 28 12:54:02.955195 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.955063    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vcl8j" event={"ID":"d3d01e36-e904-4ef1-bd3e-e2ac5b99660a","Type":"ContainerStarted","Data":"82fcb2b5d5c494e401a763244f2a7e799be3f1cad632b05ef05e186868747573"}
Jul 28 12:54:02.982576 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.982322    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s5s82" podStartSLOduration=12.882323434 podStartE2EDuration="22.982300368s" totalImagesPullingTime="10.099976934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:02.051197526 +0000 UTC m=+22.696502820" watchObservedRunningTime="2026-07-28 12:54:02.982300368 +0000 UTC m=+23.627605666"
Jul 28 12:54:02.990093 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:02.990051    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 12:54:03.758054 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:03.757877    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T12:54:02.990074415Z","UUID":"aae6785a-8934-4b30-80ca-5fb5f9827fe8","Handler":null,"Name":"","Endpoint":""}
Jul 28 12:54:03.760043 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:03.760018    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 12:54:03.760199 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:03.760066    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 12:54:03.857933 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:03.857860    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:03.858118 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:03.857999    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:03.959252 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:03.959213    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v" event={"ID":"d652d4b5-709d-42ce-89ee-6d05d54608f6","Type":"ContainerStarted","Data":"3c395ae2a25cdc8656c8d78348cffcfa33c0b749500bf98e5cdb5e25ae721d7c"}
Jul 28 12:54:03.986789 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:03.986755    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:54:03.987391 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:03.987362    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:54:04.005993 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.005944    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vcl8j" podStartSLOduration=5.410775645 podStartE2EDuration="24.005927445s" totalImagesPullingTime="18.5951518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:02.98171985 +0000 UTC m=+23.627025154" watchObservedRunningTime="2026-07-28 12:54:04.005927445 +0000 UTC m=+24.651232781"
Jul 28 12:54:04.858330 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.858295    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:04.858519 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.858335    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:04.858519 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:04.858458    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:54:04.858640 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:04.858615    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:54:04.965113 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.965069    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"381ce26ec03c6eed6f6413821545a5eb55bbe915607dd678fe8386b9fe0dac13"}
Jul 28 12:54:04.967336 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.967304    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v" event={"ID":"d652d4b5-709d-42ce-89ee-6d05d54608f6","Type":"ContainerStarted","Data":"812383b3fb6f28b5f2095b8b2beaf7e09e215111b12985060617ed6b459693ec"}
Jul 28 12:54:04.967642 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.967623    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:54:04.968205 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.968184    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s5s82"
Jul 28 12:54:04.990343 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:04.990292    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc7v" podStartSLOduration=4.020376126 podStartE2EDuration="24.990276437s" totalImagesPullingTime="20.969900311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:04.989333252 +0000 UTC m=+25.634638542" watchObservedRunningTime="2026-07-28 12:54:04.990276437 +0000 UTC m=+25.635581735"
Jul 28 12:54:05.858467 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:05.858209    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:05.858665 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:05.858573    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:06.857838 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:06.857584    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:06.858550 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:06.857584    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:06.858550 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:06.857867    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:54:06.858550 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:06.857940    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:54:06.973919 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:06.973884    2573 generic.go:356] "Generic (PLEG): container finished" podID="bea7b828-fe51-446f-a271-8900e62c55a3" containerID="f616307b625a75abb0f32761c002e0a08c8d7cf12d3823d7178500aa8ecc8a7b" exitCode=0
Jul 28 12:54:06.974068 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:06.973974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerDied","Data":"f616307b625a75abb0f32761c002e0a08c8d7cf12d3823d7178500aa8ecc8a7b"}
Jul 28 12:54:06.977680 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:06.977620    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" event={"ID":"8d74a432-2d04-4ccb-bd11-2ff510789087","Type":"ContainerStarted","Data":"fed26ded7e881e486aca484e9f47b77fbe9f4b87a1c182913ed4eca944e28ee4"}
Jul 28 12:54:06.977851 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:06.977812    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:54:07.023976 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:07.023921    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js" podStartSLOduration=8.335192307 podStartE2EDuration="27.023904423s" totalImagesPullingTime="18.688712116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:07.023767309 +0000 UTC m=+27.669072607" watchObservedRunningTime="2026-07-28 12:54:07.023904423 +0000 UTC m=+27.669209711"
Jul 28 12:54:07.858258 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:07.858220    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:07.858697 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:07.858354    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:07.980841 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:07.980799    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:54:07.981018 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:07.980891    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:54:07.998795 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:07.998763    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:54:07.999077 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:07.999059    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:54:08.578936 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.578841    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6vk6b"]
Jul 28 12:54:08.579080 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.578992    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:08.579145 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:08.579112    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:54:08.579546 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.579525    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9qcmh"]
Jul 28 12:54:08.579613 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.579602    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:08.579714 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:08.579692    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:08.581276 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.581251    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r8nsf"]
Jul 28 12:54:08.581418 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.581362    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:08.581512 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:08.581482    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:54:08.984689 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.984599    2573 generic.go:356] "Generic (PLEG): container finished" podID="bea7b828-fe51-446f-a271-8900e62c55a3" containerID="582a3d909e442903df49331e89c733ccd3d399ffe932b5731a678930b4e2d87c" exitCode=0
Jul 28 12:54:08.984689 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:08.984675    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerDied","Data":"582a3d909e442903df49331e89c733ccd3d399ffe932b5731a678930b4e2d87c"}
Jul 28 12:54:09.858293 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:09.858259    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:09.858483 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:09.858352    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:10.857858 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:10.857814    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:10.858320 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:10.857926    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:54:10.858320 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:10.857981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:10.858320 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:10.858078    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:54:10.991570 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:10.991477    2573 generic.go:356] "Generic (PLEG): container finished" podID="bea7b828-fe51-446f-a271-8900e62c55a3" containerID="33c5df38b514b31bcc7d290a228f1ede118ee674a6a5cc6a56fd8150c956d3c4" exitCode=0
Jul 28 12:54:10.991570 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:10.991542    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerDied","Data":"33c5df38b514b31bcc7d290a228f1ede118ee674a6a5cc6a56fd8150c956d3c4"}
Jul 28 12:54:11.857886 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:11.857844    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:11.858301 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:11.857970    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:12.857527 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:12.857487    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:12.857739 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:12.857487    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:12.857739 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:12.857636    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r8nsf" podUID="49c7f34f-5a6c-4986-93e4-16ff1577d508"
Jul 28 12:54:12.857739 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:12.857685    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vk6b" podUID="282de852-6184-4004-a1fd-689bd1949d36"
Jul 28 12:54:13.457448 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:13.457381    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:13.458087 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:13.457467    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:13.458087 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.457555    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:54:13.458087 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.457579    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:54:13.458087 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.457585    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:54:13.458087 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.457651    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret podName:282de852-6184-4004-a1fd-689bd1949d36 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:45.457629897 +0000 UTC m=+66.102935177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret") pod "global-pull-secret-syncer-6vk6b" (UID: "282de852-6184-4004-a1fd-689bd1949d36") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:54:13.458087 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.457589    2573 projected.go:196] Error preparing data for projected volume kube-api-access-n2cqz for pod openshift-network-diagnostics/network-check-target-9qcmh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:54:13.458087 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.457722    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz podName:3499e17e-83e5-40a6-9e3e-b1684607cfe1 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:45.457705038 +0000 UTC m=+66.103010333 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2cqz" (UniqueName: "kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz") pod "network-check-target-9qcmh" (UID: "3499e17e-83e5-40a6-9e3e-b1684607cfe1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:54:13.558555 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:13.558512    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:13.558757 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.558682    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:54:13.558757 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.558753    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs podName:49c7f34f-5a6c-4986-93e4-16ff1577d508 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:45.558733975 +0000 UTC m=+66.204039254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs") pod "network-metrics-daemon-r8nsf" (UID: "49c7f34f-5a6c-4986-93e4-16ff1577d508") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:54:13.858195 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:13.858111    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:13.858360 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:13.858249    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qcmh" podUID="3499e17e-83e5-40a6-9e3e-b1684607cfe1"
Jul 28 12:54:13.895904 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:13.895873    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sr76w_71951990-b81f-4883-94f4-cd91e45251d2/dns-node-resolver/0.log"
Jul 28 12:54:14.201582 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.201502    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeReady"
Jul 28 12:54:14.201735 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.201659    2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 12:54:14.240851 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.240817    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65b9b65cbb-jmcfn"]
Jul 28 12:54:14.258268 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.258247    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.261794 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.261773    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 12:54:14.263420 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.263379    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fzfhc\""
Jul 28 12:54:14.264689 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.264661    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 12:54:14.264884 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.264866    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 12:54:14.264949 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.264884    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hhvlv"]
Jul 28 12:54:14.271385 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.271366    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 12:54:14.284148 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.284127    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qsbvw"]
Jul 28 12:54:14.284247 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.284231    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.291506 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.289331    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 12:54:14.291506 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.289429    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 12:54:14.291506 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.289440    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gt9sx\""
Jul 28 12:54:14.291506 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.291221    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 12:54:14.302318 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.302300    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65b9b65cbb-jmcfn"]
Jul 28 12:54:14.302440 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.302330    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hhvlv"]
Jul 28 12:54:14.302440 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.302351    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qsbvw"]
Jul 28 12:54:14.302440 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.302381    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h9zqs"]
Jul 28 12:54:14.302440 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.302436    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.306186 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.305988    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 12:54:14.306186 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.305996    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-blfqk\""
Jul 28 12:54:14.306186 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.306013    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 12:54:14.306186 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.306103    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 12:54:14.306457 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.306309    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 12:54:14.320295 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.320274    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h9zqs"]
Jul 28 12:54:14.320417 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.320373    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.323089 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.323073    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xjd7z\""
Jul 28 12:54:14.323831 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.323814    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 12:54:14.324258 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.324244    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 12:54:14.365256 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365230    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-image-registry-private-configuration\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.365425 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365267    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.365425 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365311    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-bound-sa-token\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.365425 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365414    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-certificates\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.365635 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365452    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-installation-pull-secrets\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.365635 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365525    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23d6190f-829b-4fb3-9ace-72e4eab38492-ca-trust-extracted\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.365635 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365546    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfpnc\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-kube-api-access-lfpnc\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.365635 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.365588    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-trusted-ca\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.466885 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.466790    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.466885 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.466857    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-image-registry-private-configuration\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.466885 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.466885    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.466912    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgjxc\" (UniqueName: \"kubernetes.io/projected/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-api-access-lgjxc\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467043    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467124    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-bound-sa-token\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467161    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72bdd2c4-2110-465a-a54b-49c04ae93fa3-crio-socket\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467189    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72bdd2c4-2110-465a-a54b-49c04ae93fa3-data-volume\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467232    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-certificates\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.467165    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467256    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f82883b2-4aec-4ac2-a169-96e59b1cdae8-tmp-dir\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467285    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467317    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-installation-pull-secrets\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467343    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c5ch\" (UniqueName: \"kubernetes.io/projected/32827ef9-a4c6-4225-a864-097970885c5e-kube-api-access-2c5ch\") pod \"ingress-canary-hhvlv\" (UID: \"32827ef9-a4c6-4225-a864-097970885c5e\") " pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467375    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23d6190f-829b-4fb3-9ace-72e4eab38492-ca-trust-extracted\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467414    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfpnc\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-kube-api-access-lfpnc\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467438    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32827ef9-a4c6-4225-a864-097970885c5e-cert\") pod \"ingress-canary-hhvlv\" (UID: \"32827ef9-a4c6-4225-a864-097970885c5e\") " pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467482    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8clx\" (UniqueName: \"kubernetes.io/projected/f82883b2-4aec-4ac2-a169-96e59b1cdae8-kube-api-access-f8clx\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.467581 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.467260    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b9b65cbb-jmcfn: secret "image-registry-tls" not found
Jul 28 12:54:14.468245 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467703    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-trusted-ca\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.468245 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467773    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f82883b2-4aec-4ac2-a169-96e59b1cdae8-config-volume\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.468245 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.467842    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls podName:23d6190f-829b-4fb3-9ace-72e4eab38492 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:14.967819965 +0000 UTC m=+35.613125249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls") pod "image-registry-65b9b65cbb-jmcfn" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492") : secret "image-registry-tls" not found
Jul 28 12:54:14.468245 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467877    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23d6190f-829b-4fb3-9ace-72e4eab38492-ca-trust-extracted\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.468245 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.467994    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-certificates\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.468727 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.468695    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-trusted-ca\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.472698 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.472673    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-image-registry-private-configuration\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.472809 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.472677    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-installation-pull-secrets\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.478244 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.478220    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-bound-sa-token\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.479119 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.479100    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfpnc\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-kube-api-access-lfpnc\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.542440 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.542388    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-bm6p2"]
Jul 28 12:54:14.568536 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.568507    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32827ef9-a4c6-4225-a864-097970885c5e-cert\") pod \"ingress-canary-hhvlv\" (UID: \"32827ef9-a4c6-4225-a864-097970885c5e\") " pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.568714 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.568547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8clx\" (UniqueName: \"kubernetes.io/projected/f82883b2-4aec-4ac2-a169-96e59b1cdae8-kube-api-access-f8clx\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.568714 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.568574    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f82883b2-4aec-4ac2-a169-96e59b1cdae8-config-volume\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.568714 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.568601    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.568714 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.568645    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.568714 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.568699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjxc\" (UniqueName: \"kubernetes.io/projected/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-api-access-lgjxc\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.568976 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.568798    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:14.568976 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.568810    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:54:14.568976 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.568866    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:15.068845339 +0000 UTC m=+35.714150642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:14.568976 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.568948    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72bdd2c4-2110-465a-a54b-49c04ae93fa3-crio-socket\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.568976 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.568963    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls podName:f82883b2-4aec-4ac2-a169-96e59b1cdae8 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:15.068941044 +0000 UTC m=+35.714246329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls") pod "dns-default-h9zqs" (UID: "f82883b2-4aec-4ac2-a169-96e59b1cdae8") : secret "dns-default-metrics-tls" not found
Jul 28 12:54:14.569232 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569005    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72bdd2c4-2110-465a-a54b-49c04ae93fa3-data-volume\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.569232 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569052    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f82883b2-4aec-4ac2-a169-96e59b1cdae8-tmp-dir\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.569232 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569074    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72bdd2c4-2110-465a-a54b-49c04ae93fa3-crio-socket\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.569232 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569122    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.569232 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569158    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c5ch\" (UniqueName: \"kubernetes.io/projected/32827ef9-a4c6-4225-a864-097970885c5e-kube-api-access-2c5ch\") pod \"ingress-canary-hhvlv\" (UID: \"32827ef9-a4c6-4225-a864-097970885c5e\") " pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.569232 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569204    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f82883b2-4aec-4ac2-a169-96e59b1cdae8-config-volume\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.569232 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.569235    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:54:14.569533 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.569293    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:15.069277547 +0000 UTC m=+35.714582850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : configmap "kube-rbac-proxy" not found
Jul 28 12:54:14.569533 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569375    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f82883b2-4aec-4ac2-a169-96e59b1cdae8-tmp-dir\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.569533 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.569382    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72bdd2c4-2110-465a-a54b-49c04ae93fa3-data-volume\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.570529 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.570510    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-bm6p2"]
Jul 28 12:54:14.570647 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.570628    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.571459 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.571437    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32827ef9-a4c6-4225-a864-097970885c5e-cert\") pod \"ingress-canary-hhvlv\" (UID: \"32827ef9-a4c6-4225-a864-097970885c5e\") " pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.573639 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.573480    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 12:54:14.573639 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.573489    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 12:54:14.573639 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.573531    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 12:54:14.573639 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.573598    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g4qmv\""
Jul 28 12:54:14.573639 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.573619    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 12:54:14.578743 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.578719    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8clx\" (UniqueName: \"kubernetes.io/projected/f82883b2-4aec-4ac2-a169-96e59b1cdae8-kube-api-access-f8clx\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:14.579046 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.579024    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjxc\" (UniqueName: \"kubernetes.io/projected/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-api-access-lgjxc\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:14.579182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.579162    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c5ch\" (UniqueName: \"kubernetes.io/projected/32827ef9-a4c6-4225-a864-097970885c5e-kube-api-access-2c5ch\") pod \"ingress-canary-hhvlv\" (UID: \"32827ef9-a4c6-4225-a864-097970885c5e\") " pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.597299 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.597277    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hhvlv"
Jul 28 12:54:14.670410 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.670351    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/501796f5-96a3-468a-85a1-962c96a01e10-signing-key\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.670586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.670438    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z796k\" (UniqueName: \"kubernetes.io/projected/501796f5-96a3-468a-85a1-962c96a01e10-kube-api-access-z796k\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.670586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.670483    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/501796f5-96a3-468a-85a1-962c96a01e10-signing-cabundle\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.771609 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.771538    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/501796f5-96a3-468a-85a1-962c96a01e10-signing-key\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.771609 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.771591    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z796k\" (UniqueName: \"kubernetes.io/projected/501796f5-96a3-468a-85a1-962c96a01e10-kube-api-access-z796k\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.771813 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.771651    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/501796f5-96a3-468a-85a1-962c96a01e10-signing-cabundle\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.772365 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.772340    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/501796f5-96a3-468a-85a1-962c96a01e10-signing-cabundle\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.774155 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.774132    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/501796f5-96a3-468a-85a1-962c96a01e10-signing-key\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.781688 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.781664    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z796k\" (UniqueName: \"kubernetes.io/projected/501796f5-96a3-468a-85a1-962c96a01e10-kube-api-access-z796k\") pod \"service-ca-658d7b7c78-bm6p2\" (UID: \"501796f5-96a3-468a-85a1-962c96a01e10\") " pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.828956 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.828932    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bzkx5_b89a2f2a-171a-40d2-b23f-1523973a6329/node-ca/0.log"
Jul 28 12:54:14.858202 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.858175    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:14.858330 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.858208    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:14.861671 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.861647    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 12:54:14.861671 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.861658    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z7m8f\""
Jul 28 12:54:14.861831 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.861658    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 12:54:14.892164 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.892140    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2"
Jul 28 12:54:14.973182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:14.973154    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:14.973361 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.973341    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:54:14.973361 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.973361    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b9b65cbb-jmcfn: secret "image-registry-tls" not found
Jul 28 12:54:14.973507 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:14.973447    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls podName:23d6190f-829b-4fb3-9ace-72e4eab38492 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:15.973432733 +0000 UTC m=+36.618738012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls") pod "image-registry-65b9b65cbb-jmcfn" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492") : secret "image-registry-tls" not found
Jul 28 12:54:15.074192 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.073945    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:15.074317 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.074076    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:54:15.074317 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.074310    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:16.074293618 +0000 UTC m=+36.719598895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : configmap "kube-rbac-proxy" not found
Jul 28 12:54:15.074317 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.074302    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:15.074502 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.074359    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:15.074502 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.074420    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:54:15.074608 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.074549    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:15.074747 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.074726    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls podName:f82883b2-4aec-4ac2-a169-96e59b1cdae8 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:16.074712307 +0000 UTC m=+36.720017584 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls") pod "dns-default-h9zqs" (UID: "f82883b2-4aec-4ac2-a169-96e59b1cdae8") : secret "dns-default-metrics-tls" not found
Jul 28 12:54:15.074815 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.074748    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:16.074742099 +0000 UTC m=+36.720047375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:15.130798 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.130769    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hhvlv"]
Jul 28 12:54:15.133648 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.133625    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-bm6p2"]
Jul 28 12:54:15.212449 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:15.212412    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32827ef9_a4c6_4225_a864_097970885c5e.slice/crio-48df666491fec509bb2eff6ce6f48c8074bd017d9cbcf1e99ac902f2bf421085 WatchSource:0}: Error finding container 48df666491fec509bb2eff6ce6f48c8074bd017d9cbcf1e99ac902f2bf421085: Status 404 returned error can't find the container with id 48df666491fec509bb2eff6ce6f48c8074bd017d9cbcf1e99ac902f2bf421085
Jul 28 12:54:15.213422 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:15.213089    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501796f5_96a3_468a_85a1_962c96a01e10.slice/crio-1e397fb569a02095929e7a9ed1b4d7136262dacfef59d0b8b9d94d17dba03c4e WatchSource:0}: Error finding container 1e397fb569a02095929e7a9ed1b4d7136262dacfef59d0b8b9d94d17dba03c4e: Status 404 returned error can't find the container with id 1e397fb569a02095929e7a9ed1b4d7136262dacfef59d0b8b9d94d17dba03c4e
Jul 28 12:54:15.857922 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.857869    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:15.868200 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.868170    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kmztt\""
Jul 28 12:54:15.868355 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.868174    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 12:54:15.868671 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.868653    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 12:54:15.981517 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:15.981477    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:15.981702 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.981601    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:54:15.981702 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.981625    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b9b65cbb-jmcfn: secret "image-registry-tls" not found
Jul 28 12:54:15.981813 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:15.981743    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls podName:23d6190f-829b-4fb3-9ace-72e4eab38492 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:17.981719672 +0000 UTC m=+38.627024958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls") pod "image-registry-65b9b65cbb-jmcfn" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492") : secret "image-registry-tls" not found
Jul 28 12:54:16.007480 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:16.007275    2573 generic.go:356] "Generic (PLEG): container finished" podID="bea7b828-fe51-446f-a271-8900e62c55a3" containerID="06d18cf09c0d3138087d0a6010ac429e2f50119ff69d76fd21ec5973357ebecc" exitCode=0
Jul 28 12:54:16.007480 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:16.007415    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerDied","Data":"06d18cf09c0d3138087d0a6010ac429e2f50119ff69d76fd21ec5973357ebecc"}
Jul 28 12:54:16.009019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:16.008991    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hhvlv" event={"ID":"32827ef9-a4c6-4225-a864-097970885c5e","Type":"ContainerStarted","Data":"48df666491fec509bb2eff6ce6f48c8074bd017d9cbcf1e99ac902f2bf421085"}
Jul 28 12:54:16.011712 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:16.011476    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2" event={"ID":"501796f5-96a3-468a-85a1-962c96a01e10","Type":"ContainerStarted","Data":"1e397fb569a02095929e7a9ed1b4d7136262dacfef59d0b8b9d94d17dba03c4e"}
Jul 28 12:54:16.083064 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:16.083023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:16.083241 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:16.083108    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:16.083241 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:16.083158    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:16.083460 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:16.083319    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:16.083460 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:16.083385    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:18.083364444 +0000 UTC m=+38.728669722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:16.083840 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:16.083816    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:54:16.083929 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:16.083876    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:18.083860124 +0000 UTC m=+38.729165406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : configmap "kube-rbac-proxy" not found
Jul 28 12:54:16.083994 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:16.083950    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:54:16.083994 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:16.083985    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls podName:f82883b2-4aec-4ac2-a169-96e59b1cdae8 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:18.083973224 +0000 UTC m=+38.729278505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls") pod "dns-default-h9zqs" (UID: "f82883b2-4aec-4ac2-a169-96e59b1cdae8") : secret "dns-default-metrics-tls" not found
Jul 28 12:54:17.018246 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:17.018206    2573 generic.go:356] "Generic (PLEG): container finished" podID="bea7b828-fe51-446f-a271-8900e62c55a3" containerID="2ea853ac471c01df443aedcfea1352d6c7c71ef0138e2ad1c54d12ce21dd1db4" exitCode=0
Jul 28 12:54:17.018735 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:17.018277    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerDied","Data":"2ea853ac471c01df443aedcfea1352d6c7c71ef0138e2ad1c54d12ce21dd1db4"}
Jul 28 12:54:18.002789 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:18.002696    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:18.002956 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.002863    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:54:18.002956 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.002884    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b9b65cbb-jmcfn: secret "image-registry-tls" not found
Jul 28 12:54:18.002956 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.002951    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls podName:23d6190f-829b-4fb3-9ace-72e4eab38492 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:22.002931617 +0000 UTC m=+42.648236904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls") pod "image-registry-65b9b65cbb-jmcfn" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492") : secret "image-registry-tls" not found
Jul 28 12:54:18.103922 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:18.103885    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:18.103951    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:18.103994    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.104020    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.104098    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:22.104077743 +0000 UTC m=+42.749383021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : configmap "kube-rbac-proxy" not found
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.104113    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.104137    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.104151    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls podName:f82883b2-4aec-4ac2-a169-96e59b1cdae8 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:22.104141564 +0000 UTC m=+42.749446850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls") pod "dns-default-h9zqs" (UID: "f82883b2-4aec-4ac2-a169-96e59b1cdae8") : secret "dns-default-metrics-tls" not found
Jul 28 12:54:18.104356 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:18.104190    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:22.104174554 +0000 UTC m=+42.749479829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:19.027095 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:19.026856    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" event={"ID":"bea7b828-fe51-446f-a271-8900e62c55a3","Type":"ContainerStarted","Data":"c5f19fec878da4b229fe4e99ff925fbb7dffe7ae7460e17f695d087acd0803fd"}
Jul 28 12:54:19.028437 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:19.028392    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hhvlv" event={"ID":"32827ef9-a4c6-4225-a864-097970885c5e","Type":"ContainerStarted","Data":"5da72896e91c5a13d586e31d4159b6a9e2a26b59207d821c16190a98f8f73f96"}
Jul 28 12:54:19.029819 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:19.029797    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2" event={"ID":"501796f5-96a3-468a-85a1-962c96a01e10","Type":"ContainerStarted","Data":"41a53a36372c9af3a93878481810ca5f866bb7470b0c1673126fdaf0566de558"}
Jul 28 12:54:19.055465 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:19.055392    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fdhtq" podStartSLOduration=9.426567896 podStartE2EDuration="39.055374843s" totalImagesPullingTime="29.628806947s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:19.05346047 +0000 UTC m=+39.698765769" watchObservedRunningTime="2026-07-28 12:54:19.055374843 +0000 UTC m=+39.700680148"
Jul 28 12:54:19.093298 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:19.093243    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-bm6p2" podStartSLOduration=1.927407818 podStartE2EDuration="5.093226306s" totalImagesPullingTime="3.165818488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:19.07491425 +0000 UTC m=+39.720219547" watchObservedRunningTime="2026-07-28 12:54:19.093226306 +0000 UTC m=+39.738531605"
Jul 28 12:54:19.093583 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:19.093561    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hhvlv" podStartSLOduration=1.93420464 podStartE2EDuration="5.093555594s" totalImagesPullingTime="3.159350954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:19.092784403 +0000 UTC m=+39.738089693" watchObservedRunningTime="2026-07-28 12:54:19.093555594 +0000 UTC m=+39.738860892"
Jul 28 12:54:22.037620 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:22.037589    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:22.038230 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.037742    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:54:22.038230 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.037758    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65b9b65cbb-jmcfn: secret "image-registry-tls" not found
Jul 28 12:54:22.038230 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.037817    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls podName:23d6190f-829b-4fb3-9ace-72e4eab38492 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:30.037797698 +0000 UTC m=+50.683102991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls") pod "image-registry-65b9b65cbb-jmcfn" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492") : secret "image-registry-tls" not found
Jul 28 12:54:22.138172 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:22.138131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:22.138172 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:22.138188    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:22.138472 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:22.138227    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:22.138472 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.138276    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:54:22.138472 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.138358    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:22.138472 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.138385    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:54:22.138472 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.138361    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:30.138337543 +0000 UTC m=+50.783642828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : configmap "kube-rbac-proxy" not found
Jul 28 12:54:22.138472 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.138437    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls podName:72bdd2c4-2110-465a-a54b-49c04ae93fa3 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:30.138420624 +0000 UTC m=+50.783725901 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qsbvw" (UID: "72bdd2c4-2110-465a-a54b-49c04ae93fa3") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:54:22.138472 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:22.138453    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls podName:f82883b2-4aec-4ac2-a169-96e59b1cdae8 nodeName:}" failed. No retries permitted until 2026-07-28 12:54:30.138443314 +0000 UTC m=+50.783748595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls") pod "dns-default-h9zqs" (UID: "f82883b2-4aec-4ac2-a169-96e59b1cdae8") : secret "dns-default-metrics-tls" not found
Jul 28 12:54:30.102604 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.102564    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:30.106965 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.106935    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"image-registry-65b9b65cbb-jmcfn\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") " pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:30.171196 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.171159    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:30.203669 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.203639    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:30.203813 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.203691    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:30.203813 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.203735    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:30.204247 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.204222    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72bdd2c4-2110-465a-a54b-49c04ae93fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:30.206063 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.206040    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f82883b2-4aec-4ac2-a169-96e59b1cdae8-metrics-tls\") pod \"dns-default-h9zqs\" (UID: \"f82883b2-4aec-4ac2-a169-96e59b1cdae8\") " pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:30.206063 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.206050    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72bdd2c4-2110-465a-a54b-49c04ae93fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qsbvw\" (UID: \"72bdd2c4-2110-465a-a54b-49c04ae93fa3\") " pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:30.213019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.212994    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qsbvw"
Jul 28 12:54:30.230469 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.230239    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:30.319198 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.319138    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65b9b65cbb-jmcfn"]
Jul 28 12:54:30.388139 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.388102    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qsbvw"]
Jul 28 12:54:30.392506 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:30.392478    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bdd2c4_2110_465a_a54b_49c04ae93fa3.slice/crio-707e50620f3803e475b8f8f7dbf40a3b02a4cc18ba67a4801f8a81972ceeb4b1 WatchSource:0}: Error finding container 707e50620f3803e475b8f8f7dbf40a3b02a4cc18ba67a4801f8a81972ceeb4b1: Status 404 returned error can't find the container with id 707e50620f3803e475b8f8f7dbf40a3b02a4cc18ba67a4801f8a81972ceeb4b1
Jul 28 12:54:30.405219 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:30.405183    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h9zqs"]
Jul 28 12:54:30.410413 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:30.410370    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf82883b2_4aec_4ac2_a169_96e59b1cdae8.slice/crio-7001e987f430a9645cabaf86b276d2bb2a611498254bde6902b7fedad1a20c7a WatchSource:0}: Error finding container 7001e987f430a9645cabaf86b276d2bb2a611498254bde6902b7fedad1a20c7a: Status 404 returned error can't find the container with id 7001e987f430a9645cabaf86b276d2bb2a611498254bde6902b7fedad1a20c7a
Jul 28 12:54:31.061918 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:31.061881    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsbvw" event={"ID":"72bdd2c4-2110-465a-a54b-49c04ae93fa3","Type":"ContainerStarted","Data":"707e50620f3803e475b8f8f7dbf40a3b02a4cc18ba67a4801f8a81972ceeb4b1"}
Jul 28 12:54:31.063521 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:31.063492    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn" event={"ID":"23d6190f-829b-4fb3-9ace-72e4eab38492","Type":"ContainerStarted","Data":"7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46"}
Jul 28 12:54:31.063650 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:31.063546    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn" event={"ID":"23d6190f-829b-4fb3-9ace-72e4eab38492","Type":"ContainerStarted","Data":"1c5d4444a56efb1ab24233f871ab29ae8bdeb14c2b170984fcaacf8b9daef578"}
Jul 28 12:54:31.063650 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:31.063625    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:31.064661 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:31.064642    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h9zqs" event={"ID":"f82883b2-4aec-4ac2-a169-96e59b1cdae8","Type":"ContainerStarted","Data":"7001e987f430a9645cabaf86b276d2bb2a611498254bde6902b7fedad1a20c7a"}
Jul 28 12:54:31.089260 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:31.089203    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn" podStartSLOduration=29.089187279 podStartE2EDuration="29.089187279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:31.087672483 +0000 UTC m=+51.732977791" watchObservedRunningTime="2026-07-28 12:54:31.089187279 +0000 UTC m=+51.734492555"
Jul 28 12:54:33.072979 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:33.072941    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h9zqs" event={"ID":"f82883b2-4aec-4ac2-a169-96e59b1cdae8","Type":"ContainerStarted","Data":"6d20a6ed9a47288d07f3e31d52799fb7a9c4d8becf84daa4133e3190547cc1e6"}
Jul 28 12:54:33.073451 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:33.073009    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h9zqs" event={"ID":"f82883b2-4aec-4ac2-a169-96e59b1cdae8","Type":"ContainerStarted","Data":"c78f7ec9370d506437892e6a8471d501bbbcc3299dfb58174b63ff158ac3aafb"}
Jul 28 12:54:33.073451 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:33.073089    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:33.092798 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:33.092745    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h9zqs" podStartSLOduration=17.221396713 podStartE2EDuration="19.092728653s" totalImagesPullingTime="1.87133194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:33.092242358 +0000 UTC m=+53.737547660" watchObservedRunningTime="2026-07-28 12:54:33.092728653 +0000 UTC m=+53.738033951"
Jul 28 12:54:36.082643 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:36.082551    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsbvw" event={"ID":"72bdd2c4-2110-465a-a54b-49c04ae93fa3","Type":"ContainerStarted","Data":"3d50c330eeee4d4b8834813a99003cdc37214696c404b613b672883b9d05d2d9"}
Jul 28 12:54:37.592853 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.592817    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"]
Jul 28 12:54:37.654450 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.654414    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"]
Jul 28 12:54:37.654617 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.654510    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.660945 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.660623    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 12:54:37.661931 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.661006    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 12:54:37.661931 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.661222    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 12:54:37.661931 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.661718    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 12:54:37.672765 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.672736    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"]
Jul 28 12:54:37.672931 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.672783    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65b9b65cbb-jmcfn"]
Jul 28 12:54:37.672931 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.672816    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"]
Jul 28 12:54:37.672931 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.672874    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.677809 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.677782    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 12:54:37.678148 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.678126    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 12:54:37.678251 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.678168    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 12:54:37.678663 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.678643    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 12:54:37.719896 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.719864    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79979f7559-dh4c9"]
Jul 28 12:54:37.749256 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.749218    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79979f7559-dh4c9"]
Jul 28 12:54:37.749445 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.749317    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.769682 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769642    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.769862 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769703    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.769862 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769755    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-hub\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.769862 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769794    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4628fff3-b919-4279-8cb3-ba81ec7d90d7-tmp\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.769862 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769842    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.770074 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769872    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4628fff3-b919-4279-8cb3-ba81ec7d90d7-klusterlet-config\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.770074 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769922    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6v6b\" (UniqueName: \"kubernetes.io/projected/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-kube-api-access-v6v6b\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.770074 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.769998    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-ca\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.770074 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.770036    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct8mr\" (UniqueName: \"kubernetes.io/projected/4628fff3-b919-4279-8cb3-ba81ec7d90d7-kube-api-access-ct8mr\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.871302 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871202    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4628fff3-b919-4279-8cb3-ba81ec7d90d7-klusterlet-config\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.871302 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871255    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/36876a04-8485-46b2-bf64-b916527acafe-registry-certificates\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.871302 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871294    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6v6b\" (UniqueName: \"kubernetes.io/projected/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-kube-api-access-v6v6b\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871336    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/36876a04-8485-46b2-bf64-b916527acafe-image-registry-private-configuration\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-ca\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871415    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct8mr\" (UniqueName: \"kubernetes.io/projected/4628fff3-b919-4279-8cb3-ba81ec7d90d7-kube-api-access-ct8mr\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871440    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/36876a04-8485-46b2-bf64-b916527acafe-ca-trust-extracted\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871520    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871551    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-hub\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.871599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871577    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4628fff3-b919-4279-8cb3-ba81ec7d90d7-tmp\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.871963 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871605    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/36876a04-8485-46b2-bf64-b916527acafe-installation-pull-secrets\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.871963 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871629    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtzhq\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-kube-api-access-mtzhq\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.871963 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871660    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/36876a04-8485-46b2-bf64-b916527acafe-trusted-ca\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.871963 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871689    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.871963 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871712    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-bound-sa-token\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.871963 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.871735    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-registry-tls\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.872227 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.872043    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4628fff3-b919-4279-8cb3-ba81ec7d90d7-tmp\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.872839 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.872811    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.874547 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.874500    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4628fff3-b919-4279-8cb3-ba81ec7d90d7-klusterlet-config\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.874972 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.874714    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.875313 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.875273    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-hub\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.875805 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.875769    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-ca\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.876146 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.876127    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.881062 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.881024    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6v6b\" (UniqueName: \"kubernetes.io/projected/b3ee8a4c-9ec0-4a0e-ac43-df494c964682-kube-api-access-v6v6b\") pod \"cluster-proxy-proxy-agent-7cff448bd8-5schd\" (UID: \"b3ee8a4c-9ec0-4a0e-ac43-df494c964682\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:37.882589 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.882567    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct8mr\" (UniqueName: \"kubernetes.io/projected/4628fff3-b919-4279-8cb3-ba81ec7d90d7-kube-api-access-ct8mr\") pod \"klusterlet-addon-workmgr-587d5597d8-5jc58\" (UID: \"4628fff3-b919-4279-8cb3-ba81ec7d90d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.967679 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.967641    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:37.973018 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.972979    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/36876a04-8485-46b2-bf64-b916527acafe-installation-pull-secrets\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973034    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtzhq\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-kube-api-access-mtzhq\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973069    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/36876a04-8485-46b2-bf64-b916527acafe-trusted-ca\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973101    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-bound-sa-token\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973182 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973126    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-registry-tls\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973640 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973377    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/36876a04-8485-46b2-bf64-b916527acafe-registry-certificates\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973640 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973477    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/36876a04-8485-46b2-bf64-b916527acafe-image-registry-private-configuration\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973640 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973522    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/36876a04-8485-46b2-bf64-b916527acafe-ca-trust-extracted\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.973992 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.973964    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/36876a04-8485-46b2-bf64-b916527acafe-ca-trust-extracted\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.974235 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.974189    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/36876a04-8485-46b2-bf64-b916527acafe-trusted-ca\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.974235 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.974224    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/36876a04-8485-46b2-bf64-b916527acafe-registry-certificates\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.976218 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.976186    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/36876a04-8485-46b2-bf64-b916527acafe-installation-pull-secrets\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.976329 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.976227    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-registry-tls\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.976728 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.976685    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/36876a04-8485-46b2-bf64-b916527acafe-image-registry-private-configuration\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.984120 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.983166    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-bound-sa-token\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.984120 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.983739    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtzhq\" (UniqueName: \"kubernetes.io/projected/36876a04-8485-46b2-bf64-b916527acafe-kube-api-access-mtzhq\") pod \"image-registry-79979f7559-dh4c9\" (UID: \"36876a04-8485-46b2-bf64-b916527acafe\") " pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:37.993577 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:37.993545    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"
Jul 28 12:54:38.060581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:38.060546    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:39.586272 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:39.586232    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd"]
Jul 28 12:54:39.588391 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:39.588362    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3ee8a4c_9ec0_4a0e_ac43_df494c964682.slice/crio-77ba4360544e660d3495369c56fcc592efdbff55ee0dc648a41a6b8abd69010d WatchSource:0}: Error finding container 77ba4360544e660d3495369c56fcc592efdbff55ee0dc648a41a6b8abd69010d: Status 404 returned error can't find the container with id 77ba4360544e660d3495369c56fcc592efdbff55ee0dc648a41a6b8abd69010d
Jul 28 12:54:39.603726 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:39.603700    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79979f7559-dh4c9"]
Jul 28 12:54:39.607279 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:39.607255    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36876a04_8485_46b2_bf64_b916527acafe.slice/crio-e58ab458cdfdb08f5c3412d7c23b082e749216c5de46bffd2b9542dac1cba302 WatchSource:0}: Error finding container e58ab458cdfdb08f5c3412d7c23b082e749216c5de46bffd2b9542dac1cba302: Status 404 returned error can't find the container with id e58ab458cdfdb08f5c3412d7c23b082e749216c5de46bffd2b9542dac1cba302
Jul 28 12:54:39.618812 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:39.618731    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"]
Jul 28 12:54:39.620809 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:39.620790    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4628fff3_b919_4279_8cb3_ba81ec7d90d7.slice/crio-9152e005342a9dd3b69977c8ff72b7640e3186b97665c96f21ec996b23a8061e WatchSource:0}: Error finding container 9152e005342a9dd3b69977c8ff72b7640e3186b97665c96f21ec996b23a8061e: Status 404 returned error can't find the container with id 9152e005342a9dd3b69977c8ff72b7640e3186b97665c96f21ec996b23a8061e
Jul 28 12:54:39.998439 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:39.998390    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr6js"
Jul 28 12:54:40.098159 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:40.098123    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsbvw" event={"ID":"72bdd2c4-2110-465a-a54b-49c04ae93fa3","Type":"ContainerStarted","Data":"d934185059e73a7e691a5ba624446d94a8b54cf86a7d8dbb94bdc2554ed88248"}
Jul 28 12:54:40.099972 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:40.099803    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79979f7559-dh4c9" event={"ID":"36876a04-8485-46b2-bf64-b916527acafe","Type":"ContainerStarted","Data":"82f20333bdce94d382fdc0ddceb1a51babaf87ba4fa5611d4010d779771ffd03"}
Jul 28 12:54:40.099972 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:40.099866    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79979f7559-dh4c9" event={"ID":"36876a04-8485-46b2-bf64-b916527acafe","Type":"ContainerStarted","Data":"e58ab458cdfdb08f5c3412d7c23b082e749216c5de46bffd2b9542dac1cba302"}
Jul 28 12:54:40.100175 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:40.099976    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:54:40.101172 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:40.101124    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd" event={"ID":"b3ee8a4c-9ec0-4a0e-ac43-df494c964682","Type":"ContainerStarted","Data":"77ba4360544e660d3495369c56fcc592efdbff55ee0dc648a41a6b8abd69010d"}
Jul 28 12:54:40.102391 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:40.102370    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58" event={"ID":"4628fff3-b919-4279-8cb3-ba81ec7d90d7","Type":"ContainerStarted","Data":"9152e005342a9dd3b69977c8ff72b7640e3186b97665c96f21ec996b23a8061e"}
Jul 28 12:54:40.121833 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:40.121765    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79979f7559-dh4c9" podStartSLOduration=3.121746428 podStartE2EDuration="3.121746428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:40.121103033 +0000 UTC m=+60.766408334" watchObservedRunningTime="2026-07-28 12:54:40.121746428 +0000 UTC m=+60.767051726"
Jul 28 12:54:43.077678 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:43.077525    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h9zqs"
Jul 28 12:54:45.543011 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.542967    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:45.543011 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.543016    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:45.546170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.546143    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 12:54:45.546294 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.546246    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 12:54:45.557207 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.557070    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282de852-6184-4004-a1fd-689bd1949d36-original-pull-secret\") pod \"global-pull-secret-syncer-6vk6b\" (UID: \"282de852-6184-4004-a1fd-689bd1949d36\") " pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:45.558829 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.558722    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 12:54:45.567819 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.567787    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2cqz\" (UniqueName: \"kubernetes.io/projected/3499e17e-83e5-40a6-9e3e-b1684607cfe1-kube-api-access-n2cqz\") pod \"network-check-target-9qcmh\" (UID: \"3499e17e-83e5-40a6-9e3e-b1684607cfe1\") " pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:45.581659 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.581628    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kmztt\""
Jul 28 12:54:45.588581 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.588530    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:45.644444 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.644410    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:45.647943 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.647917    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 12:54:45.657543 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.657509    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49c7f34f-5a6c-4986-93e4-16ff1577d508-metrics-certs\") pod \"network-metrics-daemon-r8nsf\" (UID: \"49c7f34f-5a6c-4986-93e4-16ff1577d508\") " pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:45.771433 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.771378    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vk6b"
Jul 28 12:54:45.780754 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.780720    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z7m8f\""
Jul 28 12:54:45.788303 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:45.788267    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r8nsf"
Jul 28 12:54:46.913418 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:46.913326    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6vk6b"]
Jul 28 12:54:46.918190 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:46.918150    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282de852_6184_4004_a1fd_689bd1949d36.slice/crio-77f6229e23c429664dc4faa1354e06e53b27ad20177b41d0669d25db98aac925 WatchSource:0}: Error finding container 77f6229e23c429664dc4faa1354e06e53b27ad20177b41d0669d25db98aac925: Status 404 returned error can't find the container with id 77f6229e23c429664dc4faa1354e06e53b27ad20177b41d0669d25db98aac925
Jul 28 12:54:47.133528 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.133491    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58" event={"ID":"4628fff3-b919-4279-8cb3-ba81ec7d90d7","Type":"ContainerStarted","Data":"2a351f4da0468f6d6498012d18d132037be30ac76e9b1b49c7c89ffedae6b468"}
Jul 28 12:54:47.133742 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.133669    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:47.135677 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.135641    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qsbvw" event={"ID":"72bdd2c4-2110-465a-a54b-49c04ae93fa3","Type":"ContainerStarted","Data":"a0ac4fe3a7925d45824ff3ea77836a6bacde1983684350866cae78f1ef163f82"}
Jul 28 12:54:47.135808 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.135733    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58"
Jul 28 12:54:47.136787 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.136761    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6vk6b" event={"ID":"282de852-6184-4004-a1fd-689bd1949d36","Type":"ContainerStarted","Data":"77f6229e23c429664dc4faa1354e06e53b27ad20177b41d0669d25db98aac925"}
Jul 28 12:54:47.138758 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.138736    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd" event={"ID":"b3ee8a4c-9ec0-4a0e-ac43-df494c964682","Type":"ContainerStarted","Data":"e28024278eac34fa29bf540ae2df3fe1fb70458c1f81a773feb1252e44c5d2df"}
Jul 28 12:54:47.138862 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.138772    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd" event={"ID":"b3ee8a4c-9ec0-4a0e-ac43-df494c964682","Type":"ContainerStarted","Data":"b8cd72b1f796d413d530f61801d60d7464ef05098ef0bfb35a09e1b30b7bcd68"}
Jul 28 12:54:47.138862 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.138790    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd" event={"ID":"b3ee8a4c-9ec0-4a0e-ac43-df494c964682","Type":"ContainerStarted","Data":"b4aaf97b455f3971922aa090cc2a1bef59e9e40289ce7299a87b3865e485b34f"}
Jul 28 12:54:47.146228 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.146158    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r8nsf"]
Jul 28 12:54:47.147833 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.147798    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9qcmh"]
Jul 28 12:54:47.149837 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:47.149812    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c7f34f_5a6c_4986_93e4_16ff1577d508.slice/crio-f2b96b455bd88cef39bac01e6f2b77efba4c74b060006f3c77b65fe77af6d0e9 WatchSource:0}: Error finding container f2b96b455bd88cef39bac01e6f2b77efba4c74b060006f3c77b65fe77af6d0e9: Status 404 returned error can't find the container with id f2b96b455bd88cef39bac01e6f2b77efba4c74b060006f3c77b65fe77af6d0e9
Jul 28 12:54:47.151597 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:47.151520    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3499e17e_83e5_40a6_9e3e_b1684607cfe1.slice/crio-4dca0afc3b9db0b5f0e656bad0894386584570e6fb7ca65fb6885fc30e93dc0d WatchSource:0}: Error finding container 4dca0afc3b9db0b5f0e656bad0894386584570e6fb7ca65fb6885fc30e93dc0d: Status 404 returned error can't find the container with id 4dca0afc3b9db0b5f0e656bad0894386584570e6fb7ca65fb6885fc30e93dc0d
Jul 28 12:54:47.184724 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.184647    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-587d5597d8-5jc58" podStartSLOduration=3.049469833 podStartE2EDuration="10.184630268s" totalImagesPullingTime="7.135160435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:47.161719542 +0000 UTC m=+67.807024840" watchObservedRunningTime="2026-07-28 12:54:47.184630268 +0000 UTC m=+67.829935565"
Jul 28 12:54:47.185119 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.185084    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qsbvw" podStartSLOduration=16.973613172 podStartE2EDuration="33.185072814s" totalImagesPullingTime="16.211459642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:54:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:47.184192547 +0000 UTC m=+67.829497845" watchObservedRunningTime="2026-07-28 12:54:47.185072814 +0000 UTC m=+67.830378111"
Jul 28 12:54:47.231379 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.231321    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff448bd8-5schd" podStartSLOduration=3.081596911 podStartE2EDuration="10.231302255s" totalImagesPullingTime="7.149705344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:47.22992793 +0000 UTC m=+67.875233229" watchObservedRunningTime="2026-07-28 12:54:47.231302255 +0000 UTC m=+67.876607554"
Jul 28 12:54:47.681090 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:47.681062    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:54:48.144966 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:48.144863    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r8nsf" event={"ID":"49c7f34f-5a6c-4986-93e4-16ff1577d508","Type":"ContainerStarted","Data":"f2b96b455bd88cef39bac01e6f2b77efba4c74b060006f3c77b65fe77af6d0e9"}
Jul 28 12:54:48.146722 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:48.146689    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9qcmh" event={"ID":"3499e17e-83e5-40a6-9e3e-b1684607cfe1","Type":"ContainerStarted","Data":"4dca0afc3b9db0b5f0e656bad0894386584570e6fb7ca65fb6885fc30e93dc0d"}
Jul 28 12:54:49.154852 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:49.154720    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r8nsf" event={"ID":"49c7f34f-5a6c-4986-93e4-16ff1577d508","Type":"ContainerStarted","Data":"7656c3f416104a7528c2da086c31251671f956b44dcaa3fa2dfba9b236d91582"}
Jul 28 12:54:49.154852 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:49.154813    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r8nsf" event={"ID":"49c7f34f-5a6c-4986-93e4-16ff1577d508","Type":"ContainerStarted","Data":"1cb74fbdd6d0a1a3e2a9527a4f1bbe4fccb1fb89e17130fa8ffceec5a5f095d3"}
Jul 28 12:54:49.173285 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:49.173208    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r8nsf" podStartSLOduration=68.13358844 podStartE2EDuration="1m9.173184333s" totalImagesPullingTime="1.039595893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:49.172268965 +0000 UTC m=+69.817574263" watchObservedRunningTime="2026-07-28 12:54:49.173184333 +0000 UTC m=+69.818489631"
Jul 28 12:54:51.599500 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.598179    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zdbdl"]
Jul 28 12:54:51.632071 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.631687    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.646933 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.646903    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 12:54:51.647911 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.647323    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 12:54:51.647911 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.647502    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 12:54:51.647911 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.647631    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 12:54:51.647911 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.647820    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 12:54:51.648500 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.648230    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bjvgr\""
Jul 28 12:54:51.648500 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.648371    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 12:54:51.801829 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.801777    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-textfile\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.801829 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.801833    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-wtmp\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.802073 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.801943    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-root\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.802073 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.802007    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-accelerators-collector-config\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.802073 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.802048    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-metrics-client-ca\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.802209 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.802075    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-sys\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.802209 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.802114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.802209 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.802166    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s87zc\" (UniqueName: \"kubernetes.io/projected/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-kube-api-access-s87zc\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.802352 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.802217    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-tls\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903545 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903458    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-metrics-client-ca\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903545 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903507    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-sys\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903766 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903562    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903820 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903784    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-sys\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903871 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903843    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s87zc\" (UniqueName: \"kubernetes.io/projected/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-kube-api-access-s87zc\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903920 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903878    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-tls\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903970 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903916    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-textfile\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.903970 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903944    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-wtmp\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.904066 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.903986    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-root\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.904066 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:51.904037    2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 12:54:51.904159 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:54:51.904120    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-tls podName:06e3413b-1a24-4f95-b0cc-0d02c0bde21a nodeName:}" failed. No retries permitted until 2026-07-28 12:54:52.404082705 +0000 UTC m=+73.049387987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-tls") pod "node-exporter-zdbdl" (UID: "06e3413b-1a24-4f95-b0cc-0d02c0bde21a") : secret "node-exporter-tls" not found
Jul 28 12:54:51.904263 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.904235    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-metrics-client-ca\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.904591 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.904416    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-wtmp\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.904591 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.904515    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-accelerators-collector-config\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.904591 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.904034    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-accelerators-collector-config\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.904829 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.904571    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-root\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.905300 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.905245    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-textfile\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.915822 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.915763    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s87zc\" (UniqueName: \"kubernetes.io/projected/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-kube-api-access-s87zc\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:51.920575 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:51.920540    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:52.409920 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:52.409453    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-tls\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:52.413438 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:52.413357    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06e3413b-1a24-4f95-b0cc-0d02c0bde21a-node-exporter-tls\") pod \"node-exporter-zdbdl\" (UID: \"06e3413b-1a24-4f95-b0cc-0d02c0bde21a\") " pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:52.557150 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:52.557118    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zdbdl"
Jul 28 12:54:52.567124 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:54:52.567084    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e3413b_1a24_4f95_b0cc_0d02c0bde21a.slice/crio-f028bf7b28fae3463ccc3966ff324c6ebb36c7ab397f0474939cae18db92a675 WatchSource:0}: Error finding container f028bf7b28fae3463ccc3966ff324c6ebb36c7ab397f0474939cae18db92a675: Status 404 returned error can't find the container with id f028bf7b28fae3463ccc3966ff324c6ebb36c7ab397f0474939cae18db92a675
Jul 28 12:54:53.171828 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:53.171790    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdbdl" event={"ID":"06e3413b-1a24-4f95-b0cc-0d02c0bde21a","Type":"ContainerStarted","Data":"f028bf7b28fae3463ccc3966ff324c6ebb36c7ab397f0474939cae18db92a675"}
Jul 28 12:54:53.173211 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:53.173185    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6vk6b" event={"ID":"282de852-6184-4004-a1fd-689bd1949d36","Type":"ContainerStarted","Data":"e6b67d1232e6e366a5ced7a146319ae6accdfc80e4f5e6449cc6a2d2c2eb6207"}
Jul 28 12:54:53.174612 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:53.174592    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9qcmh" event={"ID":"3499e17e-83e5-40a6-9e3e-b1684607cfe1","Type":"ContainerStarted","Data":"c9d5379c2d0493cd3792ba81a7052a54aeda985c9df31a8b80e96d5b9edc8434"}
Jul 28 12:54:53.174763 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:53.174749    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:54:53.190603 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:53.190552    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6vk6b" podStartSLOduration=67.774412911 podStartE2EDuration="1m13.190536719s" totalImagesPullingTime="5.416123808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:53.190294604 +0000 UTC m=+73.835599905" watchObservedRunningTime="2026-07-28 12:54:53.190536719 +0000 UTC m=+73.835842016"
Jul 28 12:54:53.206700 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:53.206632    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9qcmh" podStartSLOduration=68.013752478 podStartE2EDuration="1m13.206611598s" totalImagesPullingTime="5.19285912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:53.205882842 +0000 UTC m=+73.851188141" watchObservedRunningTime="2026-07-28 12:54:53.206611598 +0000 UTC m=+73.851916896"
Jul 28 12:54:54.179477 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:54.179438    2573 generic.go:356] "Generic (PLEG): container finished" podID="06e3413b-1a24-4f95-b0cc-0d02c0bde21a" containerID="5a8273ccec7c858f8eb9d3d04b7805539028c70e6be3344b0c18bb2e63c656c0" exitCode=0
Jul 28 12:54:54.179945 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:54.179521    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdbdl" event={"ID":"06e3413b-1a24-4f95-b0cc-0d02c0bde21a","Type":"ContainerDied","Data":"5a8273ccec7c858f8eb9d3d04b7805539028c70e6be3344b0c18bb2e63c656c0"}
Jul 28 12:54:55.184754 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:55.184708    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdbdl" event={"ID":"06e3413b-1a24-4f95-b0cc-0d02c0bde21a","Type":"ContainerStarted","Data":"7c39647d1e704682020903a9d573af271329a5ea50d1d43ab7257c1884be04bb"}
Jul 28 12:54:55.185170 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:55.184817    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdbdl" event={"ID":"06e3413b-1a24-4f95-b0cc-0d02c0bde21a","Type":"ContainerStarted","Data":"d2a9b153d208c939893420d2fe28a2437545fca535dc2d67b77174d1a35396e2"}
Jul 28 12:54:55.210879 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:54:55.210828    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zdbdl" podStartSLOduration=3.313288672 podStartE2EDuration="4.210810991s" totalImagesPullingTime="825.952596ms" totalInitContainerRuntime="71.569723ms" isStatefulPod=true podCreationTimestamp="2026-07-28 12:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:55.21052697 +0000 UTC m=+75.855832267" watchObservedRunningTime="2026-07-28 12:54:55.210810991 +0000 UTC m=+75.856116289"
Jul 28 12:55:01.113907 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:01.113875    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79979f7559-dh4c9"
Jul 28 12:55:02.696007 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:02.695949    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn" podUID="23d6190f-829b-4fb3-9ace-72e4eab38492" containerName="registry" containerID="cri-o://7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46" gracePeriod=30
Jul 28 12:55:02.935026 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:02.935000    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:55:03.087970 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.087939    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-certificates\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.087994    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfpnc\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-kube-api-access-lfpnc\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.088027    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-trusted-ca\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.088053    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.088103    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23d6190f-829b-4fb3-9ace-72e4eab38492-ca-trust-extracted\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.088138    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-installation-pull-secrets\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.088168    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-bound-sa-token\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088201 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.088198    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-image-registry-private-configuration\") pod \"23d6190f-829b-4fb3-9ace-72e4eab38492\" (UID: \"23d6190f-829b-4fb3-9ace-72e4eab38492\") "
Jul 28 12:55:03.088563 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.088524    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:55:03.089040 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.089011    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:55:03.090637 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.090607    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:55:03.090753 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.090637    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:55:03.090810 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.090749    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:55:03.090873 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.090849    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-kube-api-access-lfpnc" (OuterVolumeSpecName: "kube-api-access-lfpnc") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "kube-api-access-lfpnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:55:03.090922 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.090906    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:55:03.097107 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.097076    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d6190f-829b-4fb3-9ace-72e4eab38492-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "23d6190f-829b-4fb3-9ace-72e4eab38492" (UID: "23d6190f-829b-4fb3-9ace-72e4eab38492"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:55:03.188948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188910    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-trusted-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.188948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188939    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.188948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188947    2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23d6190f-829b-4fb3-9ace-72e4eab38492-ca-trust-extracted\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.188948 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188958    2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-installation-pull-secrets\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.189205 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188967    2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-bound-sa-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.189205 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188977    2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23d6190f-829b-4fb3-9ace-72e4eab38492-image-registry-private-configuration\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.189205 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188986    2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23d6190f-829b-4fb3-9ace-72e4eab38492-registry-certificates\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.189205 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.188994    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfpnc\" (UniqueName: \"kubernetes.io/projected/23d6190f-829b-4fb3-9ace-72e4eab38492-kube-api-access-lfpnc\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:03.213097 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.213058    2573 generic.go:356] "Generic (PLEG): container finished" podID="23d6190f-829b-4fb3-9ace-72e4eab38492" containerID="7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46" exitCode=0
Jul 28 12:55:03.213239 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.213109    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn"
Jul 28 12:55:03.213239 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.213144    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn" event={"ID":"23d6190f-829b-4fb3-9ace-72e4eab38492","Type":"ContainerDied","Data":"7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46"}
Jul 28 12:55:03.213239 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.213200    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65b9b65cbb-jmcfn" event={"ID":"23d6190f-829b-4fb3-9ace-72e4eab38492","Type":"ContainerDied","Data":"1c5d4444a56efb1ab24233f871ab29ae8bdeb14c2b170984fcaacf8b9daef578"}
Jul 28 12:55:03.213239 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.213223    2573 scope.go:117] "RemoveContainer" containerID="7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46"
Jul 28 12:55:03.221788 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.221772    2573 scope.go:117] "RemoveContainer" containerID="7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46"
Jul 28 12:55:03.222087 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:55:03.222067    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46\": container with ID starting with 7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46 not found: ID does not exist" containerID="7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46"
Jul 28 12:55:03.222129 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.222095    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46"} err="failed to get container status \"7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46\": rpc error: code = NotFound desc = could not find container \"7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46\": container with ID starting with 7d9d786fadf63715fed2507a0183301133ecc60ad175cd7b487fbaf194b31d46 not found: ID does not exist"
Jul 28 12:55:03.236295 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.236265    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65b9b65cbb-jmcfn"]
Jul 28 12:55:03.239970 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.239930    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65b9b65cbb-jmcfn"]
Jul 28 12:55:03.862671 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:03.862632    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23d6190f-829b-4fb3-9ace-72e4eab38492" path="/var/lib/kubelet/pods/23d6190f-829b-4fb3-9ace-72e4eab38492/volumes"
Jul 28 12:55:06.259101 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.259060    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66b6b85465-fstc6"]
Jul 28 12:55:06.259586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.259388    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d6190f-829b-4fb3-9ace-72e4eab38492" containerName="registry"
Jul 28 12:55:06.259586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.259421    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d6190f-829b-4fb3-9ace-72e4eab38492" containerName="registry"
Jul 28 12:55:06.259586 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.259489    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d6190f-829b-4fb3-9ace-72e4eab38492" containerName="registry"
Jul 28 12:55:06.265353 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.265329    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.269640 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.269613    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 12:55:06.269640 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.269630    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 12:55:06.269640 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.269630    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5tvcp\""
Jul 28 12:55:06.269910 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.269772    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 12:55:06.269910 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.269771    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 12:55:06.269910 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.269795    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 12:55:06.270025 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.270010    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 12:55:06.270070 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.270010    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 12:55:06.273866 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.273839    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b6b85465-fstc6"]
Jul 28 12:55:06.411649 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.411615    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpnvc\" (UniqueName: \"kubernetes.io/projected/d0d346bf-b5f5-4ca7-802a-4313cf687716-kube-api-access-vpnvc\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.411810 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.411658    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-oauth-config\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.411810 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.411675    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-config\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.411810 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.411707    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-service-ca\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.411810 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.411770    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-serving-cert\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.411810 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.411806    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-oauth-serving-cert\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.513122 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513028    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-oauth-config\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.513122 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513069    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-config\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.513122 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-service-ca\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.513426 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-serving-cert\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.513426 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513150    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-oauth-serving-cert\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.513426 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513174    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpnvc\" (UniqueName: \"kubernetes.io/projected/d0d346bf-b5f5-4ca7-802a-4313cf687716-kube-api-access-vpnvc\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.513911 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513885    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-service-ca\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.514017 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513914    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-config\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.514017 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.513939    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-oauth-serving-cert\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.515501 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.515479    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-oauth-config\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.515635 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.515618    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-serving-cert\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.523195 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.523175    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpnvc\" (UniqueName: \"kubernetes.io/projected/d0d346bf-b5f5-4ca7-802a-4313cf687716-kube-api-access-vpnvc\") pod \"console-66b6b85465-fstc6\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") " pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.575146 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.575108    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:06.704375 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:06.704287    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b6b85465-fstc6"]
Jul 28 12:55:06.706920 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:55:06.706894    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d346bf_b5f5_4ca7_802a_4313cf687716.slice/crio-4f62b62a68504af9a537bbfb33ab750e081069d9471caa1e9542256de27b9aa7 WatchSource:0}: Error finding container 4f62b62a68504af9a537bbfb33ab750e081069d9471caa1e9542256de27b9aa7: Status 404 returned error can't find the container with id 4f62b62a68504af9a537bbfb33ab750e081069d9471caa1e9542256de27b9aa7
Jul 28 12:55:07.228178 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:07.228145    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6b85465-fstc6" event={"ID":"d0d346bf-b5f5-4ca7-802a-4313cf687716","Type":"ContainerStarted","Data":"4f62b62a68504af9a537bbfb33ab750e081069d9471caa1e9542256de27b9aa7"}
Jul 28 12:55:10.242008 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:10.241971    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6b85465-fstc6" event={"ID":"d0d346bf-b5f5-4ca7-802a-4313cf687716","Type":"ContainerStarted","Data":"5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147"}
Jul 28 12:55:11.267376 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:11.267316    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66b6b85465-fstc6" podStartSLOduration=1.832324471 podStartE2EDuration="5.267298938s" totalImagesPullingTime="3.434974467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:55:11.266513106 +0000 UTC m=+91.911818405" watchObservedRunningTime="2026-07-28 12:55:11.267298938 +0000 UTC m=+91.912604237"
Jul 28 12:55:14.836078 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:14.836043    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-555ffdd789-x5g6z"]
Jul 28 12:55:15.025252 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.025212    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555ffdd789-x5g6z"]
Jul 28 12:55:15.025459 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.025307    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.034299 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.034275    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 12:55:15.076782 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.076746    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-trusted-ca-bundle\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.076782 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.076781    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-oauth-serving-cert\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.076997 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.076859    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-config\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.076997 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.076926    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-oauth-config\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.076997 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.076959    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmgzm\" (UniqueName: \"kubernetes.io/projected/b1fbe382-afea-49b4-a9d5-dca7fb82088b-kube-api-access-qmgzm\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.077117 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.077001    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-service-ca\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.077151 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.077104    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-serving-cert\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.177525 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.177447    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-oauth-config\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.177525 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.177486    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmgzm\" (UniqueName: \"kubernetes.io/projected/b1fbe382-afea-49b4-a9d5-dca7fb82088b-kube-api-access-qmgzm\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.177525 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.177512    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-service-ca\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.177780 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.177551    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-serving-cert\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.177780 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.177585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-trusted-ca-bundle\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.177780 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.177606    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-oauth-serving-cert\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.177780 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.177639    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-config\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.178445 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.178415    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-config\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.178601 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.178422    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-service-ca\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.178601 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.178476    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-oauth-serving-cert\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.178601 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.178556    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-trusted-ca-bundle\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.179971 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.179948    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-oauth-config\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.180206 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.180185    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-serving-cert\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.187972 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.187946    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmgzm\" (UniqueName: \"kubernetes.io/projected/b1fbe382-afea-49b4-a9d5-dca7fb82088b-kube-api-access-qmgzm\") pod \"console-555ffdd789-x5g6z\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") " pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.334470 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.334430    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:15.495859 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:15.495823    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555ffdd789-x5g6z"]
Jul 28 12:55:15.498948 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:55:15.498920    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1fbe382_afea_49b4_a9d5_dca7fb82088b.slice/crio-d6867990c6c105704f6e2cfe6cfaf7e14643bd26720d8f82676e728e540d7e61 WatchSource:0}: Error finding container d6867990c6c105704f6e2cfe6cfaf7e14643bd26720d8f82676e728e540d7e61: Status 404 returned error can't find the container with id d6867990c6c105704f6e2cfe6cfaf7e14643bd26720d8f82676e728e540d7e61
Jul 28 12:55:16.263309 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:16.263271    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555ffdd789-x5g6z" event={"ID":"b1fbe382-afea-49b4-a9d5-dca7fb82088b","Type":"ContainerStarted","Data":"11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1"}
Jul 28 12:55:16.263706 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:16.263321    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555ffdd789-x5g6z" event={"ID":"b1fbe382-afea-49b4-a9d5-dca7fb82088b","Type":"ContainerStarted","Data":"d6867990c6c105704f6e2cfe6cfaf7e14643bd26720d8f82676e728e540d7e61"}
Jul 28 12:55:16.291825 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:16.291774    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-555ffdd789-x5g6z" podStartSLOduration=2.291758355 podStartE2EDuration="2.291758355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:55:16.286645587 +0000 UTC m=+96.931950909" watchObservedRunningTime="2026-07-28 12:55:16.291758355 +0000 UTC m=+96.937063652"
Jul 28 12:55:16.575646 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:16.575557    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:16.575646 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:16.575636    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:16.580574 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:16.580550    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:17.271208 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:17.271174    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:24.182251 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:24.182211    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9qcmh"
Jul 28 12:55:25.335040 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:25.334996    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:25.335513 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:25.335063    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:25.339997 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:25.339971    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:26.302263 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:26.302232    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:55:26.354420 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:26.354374    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b6b85465-fstc6"]
Jul 28 12:55:51.375905 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.375855    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66b6b85465-fstc6" podUID="d0d346bf-b5f5-4ca7-802a-4313cf687716" containerName="console" containerID="cri-o://5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147" gracePeriod=15
Jul 28 12:55:51.613745 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.613721    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:51.767174 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767125    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpnvc\" (UniqueName: \"kubernetes.io/projected/d0d346bf-b5f5-4ca7-802a-4313cf687716-kube-api-access-vpnvc\") pod \"d0d346bf-b5f5-4ca7-802a-4313cf687716\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") "
Jul 28 12:55:51.767345 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767200    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-service-ca\") pod \"d0d346bf-b5f5-4ca7-802a-4313cf687716\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") "
Jul 28 12:55:51.767345 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767228    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-oauth-serving-cert\") pod \"d0d346bf-b5f5-4ca7-802a-4313cf687716\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") "
Jul 28 12:55:51.767345 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767286    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-oauth-config\") pod \"d0d346bf-b5f5-4ca7-802a-4313cf687716\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") "
Jul 28 12:55:51.767345 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767326    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-serving-cert\") pod \"d0d346bf-b5f5-4ca7-802a-4313cf687716\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") "
Jul 28 12:55:51.767584 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767368    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-config\") pod \"d0d346bf-b5f5-4ca7-802a-4313cf687716\" (UID: \"d0d346bf-b5f5-4ca7-802a-4313cf687716\") "
Jul 28 12:55:51.767791 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767738    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d0d346bf-b5f5-4ca7-802a-4313cf687716" (UID: "d0d346bf-b5f5-4ca7-802a-4313cf687716"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:55:51.767791 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767750    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-service-ca" (OuterVolumeSpecName: "service-ca") pod "d0d346bf-b5f5-4ca7-802a-4313cf687716" (UID: "d0d346bf-b5f5-4ca7-802a-4313cf687716"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:55:51.767939 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.767845    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-config" (OuterVolumeSpecName: "console-config") pod "d0d346bf-b5f5-4ca7-802a-4313cf687716" (UID: "d0d346bf-b5f5-4ca7-802a-4313cf687716"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:55:51.769591 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.769555    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d0d346bf-b5f5-4ca7-802a-4313cf687716" (UID: "d0d346bf-b5f5-4ca7-802a-4313cf687716"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:55:51.769677 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.769611    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d0d346bf-b5f5-4ca7-802a-4313cf687716" (UID: "d0d346bf-b5f5-4ca7-802a-4313cf687716"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:55:51.769677 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.769621    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d346bf-b5f5-4ca7-802a-4313cf687716-kube-api-access-vpnvc" (OuterVolumeSpecName: "kube-api-access-vpnvc") pod "d0d346bf-b5f5-4ca7-802a-4313cf687716" (UID: "d0d346bf-b5f5-4ca7-802a-4313cf687716"). InnerVolumeSpecName "kube-api-access-vpnvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:55:51.868871 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.868836    2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:51.868871 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.868869    2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:51.869063 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.868882    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpnvc\" (UniqueName: \"kubernetes.io/projected/d0d346bf-b5f5-4ca7-802a-4313cf687716-kube-api-access-vpnvc\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:51.869063 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.868893    2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-service-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:51.869063 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.868905    2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d346bf-b5f5-4ca7-802a-4313cf687716-oauth-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:51.869063 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:51.868915    2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d346bf-b5f5-4ca7-802a-4313cf687716-console-oauth-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:55:52.391142 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.391108    2573 generic.go:356] "Generic (PLEG): container finished" podID="d0d346bf-b5f5-4ca7-802a-4313cf687716" containerID="5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147" exitCode=0
Jul 28 12:55:52.391567 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.391154    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b6b85465-fstc6"
Jul 28 12:55:52.391567 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.391181    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6b85465-fstc6" event={"ID":"d0d346bf-b5f5-4ca7-802a-4313cf687716","Type":"ContainerDied","Data":"5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147"}
Jul 28 12:55:52.391567 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.391223    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b6b85465-fstc6" event={"ID":"d0d346bf-b5f5-4ca7-802a-4313cf687716","Type":"ContainerDied","Data":"4f62b62a68504af9a537bbfb33ab750e081069d9471caa1e9542256de27b9aa7"}
Jul 28 12:55:52.391567 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.391243    2573 scope.go:117] "RemoveContainer" containerID="5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147"
Jul 28 12:55:52.399318 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.399289    2573 scope.go:117] "RemoveContainer" containerID="5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147"
Jul 28 12:55:52.399645 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:55:52.399612    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147\": container with ID starting with 5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147 not found: ID does not exist" containerID="5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147"
Jul 28 12:55:52.399750 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.399653    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147"} err="failed to get container status \"5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147\": rpc error: code = NotFound desc = could not find container \"5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147\": container with ID starting with 5d74e2049ee879d955b11002a13ef60bca71124cb69fe004f1404ec679cfd147 not found: ID does not exist"
Jul 28 12:55:52.409463 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.409430    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b6b85465-fstc6"]
Jul 28 12:55:52.413683 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:52.413660    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66b6b85465-fstc6"]
Jul 28 12:55:53.862218 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:55:53.862188    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0d346bf-b5f5-4ca7-802a-4313cf687716" path="/var/lib/kubelet/pods/d0d346bf-b5f5-4ca7-802a-4313cf687716/volumes"
Jul 28 12:56:20.220675 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.220644    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fccb4cd67-ltnhd"]
Jul 28 12:56:20.221052 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.220919    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d346bf-b5f5-4ca7-802a-4313cf687716" containerName="console"
Jul 28 12:56:20.221052 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.220937    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d346bf-b5f5-4ca7-802a-4313cf687716" containerName="console"
Jul 28 12:56:20.221052 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.220981    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d346bf-b5f5-4ca7-802a-4313cf687716" containerName="console"
Jul 28 12:56:20.223835 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.223814    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.240014 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.239986    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fccb4cd67-ltnhd"]
Jul 28 12:56:20.271587 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.271563    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnvkh\" (UniqueName: \"kubernetes.io/projected/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-kube-api-access-fnvkh\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.271708 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.271613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-oauth-serving-cert\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.271708 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.271657    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-oauth-config\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.271708 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.271680    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-service-ca\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.271708 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.271701    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-trusted-ca-bundle\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.271863 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.271782    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-config\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.271863 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.271846    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-serving-cert\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373021 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.372993    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-oauth-config\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373156 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373032    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-service-ca\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373156 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373047    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-trusted-ca-bundle\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373227 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373170    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-config\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373261 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373232    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-serving-cert\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373299 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373261    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvkh\" (UniqueName: \"kubernetes.io/projected/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-kube-api-access-fnvkh\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373338 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373320    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-oauth-serving-cert\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.373986 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373962    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-config\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.374096 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.373998    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-service-ca\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.374096 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.374035    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-oauth-serving-cert\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.374305 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.374285    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-trusted-ca-bundle\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.375737 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.375718    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-oauth-config\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.375807 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.375790    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-serving-cert\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.381337 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.381317    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnvkh\" (UniqueName: \"kubernetes.io/projected/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-kube-api-access-fnvkh\") pod \"console-7fccb4cd67-ltnhd\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") " pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.533707 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.533680    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:20.657231 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:20.657205    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fccb4cd67-ltnhd"]
Jul 28 12:56:20.659564 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:56:20.659540    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761fef6a_9b9f_4cfc_b2a1_926d66d8cb8e.slice/crio-a2a2deab19edb3b7c5d22019a138af595ccd2e198db6f57842615a22fe52e687 WatchSource:0}: Error finding container a2a2deab19edb3b7c5d22019a138af595ccd2e198db6f57842615a22fe52e687: Status 404 returned error can't find the container with id a2a2deab19edb3b7c5d22019a138af595ccd2e198db6f57842615a22fe52e687
Jul 28 12:56:21.493497 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:21.493462    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fccb4cd67-ltnhd" event={"ID":"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e","Type":"ContainerStarted","Data":"0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e"}
Jul 28 12:56:21.493857 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:21.493514    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fccb4cd67-ltnhd" event={"ID":"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e","Type":"ContainerStarted","Data":"a2a2deab19edb3b7c5d22019a138af595ccd2e198db6f57842615a22fe52e687"}
Jul 28 12:56:30.534018 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:30.533975    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:30.534449 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:30.534040    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:30.539007 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:30.538975    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:30.565543 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:30.565486    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fccb4cd67-ltnhd" podStartSLOduration=10.565470774 podStartE2EDuration="10.565470774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:56:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:56:21.512068668 +0000 UTC m=+162.157373969" watchObservedRunningTime="2026-07-28 12:56:30.565470774 +0000 UTC m=+171.210776072"
Jul 28 12:56:31.536331 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:31.536260    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:56:31.597738 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:31.597692    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-555ffdd789-x5g6z"]
Jul 28 12:56:56.616668 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.616603    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-555ffdd789-x5g6z" podUID="b1fbe382-afea-49b4-a9d5-dca7fb82088b" containerName="console" containerID="cri-o://11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1" gracePeriod=15
Jul 28 12:56:56.863259 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.863224    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:56:56.934785 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.934691    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-config\") pod \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") "
Jul 28 12:56:56.934785 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.934754    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmgzm\" (UniqueName: \"kubernetes.io/projected/b1fbe382-afea-49b4-a9d5-dca7fb82088b-kube-api-access-qmgzm\") pod \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") "
Jul 28 12:56:56.934785 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.934779    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-serving-cert\") pod \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") "
Jul 28 12:56:56.935019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.934887    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-trusted-ca-bundle\") pod \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") "
Jul 28 12:56:56.935019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.934920    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-oauth-serving-cert\") pod \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") "
Jul 28 12:56:56.935019 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.934945    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-oauth-config\") pod \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") "
Jul 28 12:56:56.935231 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.935021    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-service-ca\") pod \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\" (UID: \"b1fbe382-afea-49b4-a9d5-dca7fb82088b\") "
Jul 28 12:56:56.935289 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.935239    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-config" (OuterVolumeSpecName: "console-config") pod "b1fbe382-afea-49b4-a9d5-dca7fb82088b" (UID: "b1fbe382-afea-49b4-a9d5-dca7fb82088b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:56:56.935347 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.935283    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b1fbe382-afea-49b4-a9d5-dca7fb82088b" (UID: "b1fbe382-afea-49b4-a9d5-dca7fb82088b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:56:56.935347 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.935295    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b1fbe382-afea-49b4-a9d5-dca7fb82088b" (UID: "b1fbe382-afea-49b4-a9d5-dca7fb82088b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:56:56.935573 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.935553    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-service-ca" (OuterVolumeSpecName: "service-ca") pod "b1fbe382-afea-49b4-a9d5-dca7fb82088b" (UID: "b1fbe382-afea-49b4-a9d5-dca7fb82088b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:56:56.936977 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.936952    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1fbe382-afea-49b4-a9d5-dca7fb82088b-kube-api-access-qmgzm" (OuterVolumeSpecName: "kube-api-access-qmgzm") pod "b1fbe382-afea-49b4-a9d5-dca7fb82088b" (UID: "b1fbe382-afea-49b4-a9d5-dca7fb82088b"). InnerVolumeSpecName "kube-api-access-qmgzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:56:56.937079 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.936981    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b1fbe382-afea-49b4-a9d5-dca7fb82088b" (UID: "b1fbe382-afea-49b4-a9d5-dca7fb82088b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:56:56.937146 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:56.937100    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b1fbe382-afea-49b4-a9d5-dca7fb82088b" (UID: "b1fbe382-afea-49b4-a9d5-dca7fb82088b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:56:57.036453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.036375    2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-service-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:56:57.036453 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.036448    2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:56:57.036674 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.036465    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmgzm\" (UniqueName: \"kubernetes.io/projected/b1fbe382-afea-49b4-a9d5-dca7fb82088b-kube-api-access-qmgzm\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:56:57.036674 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.036479    2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:56:57.036674 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.036493    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-trusted-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:56:57.036674 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.036506    2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1fbe382-afea-49b4-a9d5-dca7fb82088b-oauth-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:56:57.036674 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.036518    2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1fbe382-afea-49b4-a9d5-dca7fb82088b-console-oauth-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:56:57.629633 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.629594    2573 generic.go:356] "Generic (PLEG): container finished" podID="b1fbe382-afea-49b4-a9d5-dca7fb82088b" containerID="11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1" exitCode=0
Jul 28 12:56:57.630081 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.629640    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555ffdd789-x5g6z"
Jul 28 12:56:57.630081 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.629653    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555ffdd789-x5g6z" event={"ID":"b1fbe382-afea-49b4-a9d5-dca7fb82088b","Type":"ContainerDied","Data":"11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1"}
Jul 28 12:56:57.630081 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.629704    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555ffdd789-x5g6z" event={"ID":"b1fbe382-afea-49b4-a9d5-dca7fb82088b","Type":"ContainerDied","Data":"d6867990c6c105704f6e2cfe6cfaf7e14643bd26720d8f82676e728e540d7e61"}
Jul 28 12:56:57.630081 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.629750    2573 scope.go:117] "RemoveContainer" containerID="11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1"
Jul 28 12:56:57.637865 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.637842    2573 scope.go:117] "RemoveContainer" containerID="11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1"
Jul 28 12:56:57.638120 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:56:57.638100    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1\": container with ID starting with 11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1 not found: ID does not exist" containerID="11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1"
Jul 28 12:56:57.638186 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.638135    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1"} err="failed to get container status \"11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1\": rpc error: code = NotFound desc = could not find container \"11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1\": container with ID starting with 11cdbe35bdcadcb3fd7ac1d85429497d8dce9b6de873242a519981f2faca94d1 not found: ID does not exist"
Jul 28 12:56:57.652110 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.652075    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-555ffdd789-x5g6z"]
Jul 28 12:56:57.655554 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.655529    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-555ffdd789-x5g6z"]
Jul 28 12:56:57.862197 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:56:57.862162    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1fbe382-afea-49b4-a9d5-dca7fb82088b" path="/var/lib/kubelet/pods/b1fbe382-afea-49b4-a9d5-dca7fb82088b/volumes"
Jul 28 12:57:35.216970 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.216931    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c985c5888-wmxxj"]
Jul 28 12:57:35.217477 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.217178    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1fbe382-afea-49b4-a9d5-dca7fb82088b" containerName="console"
Jul 28 12:57:35.217477 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.217190    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1fbe382-afea-49b4-a9d5-dca7fb82088b" containerName="console"
Jul 28 12:57:35.217477 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.217227    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1fbe382-afea-49b4-a9d5-dca7fb82088b" containerName="console"
Jul 28 12:57:35.219880 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.219863    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.232595 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.232569    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c985c5888-wmxxj"]
Jul 28 12:57:35.323807 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.323761    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-serving-cert\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.323807 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.323810    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9jvr\" (UniqueName: \"kubernetes.io/projected/5131399e-939c-4528-8373-f83bc1447778-kube-api-access-d9jvr\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.324026 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.323885    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-service-ca\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.324026 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.323917    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-oauth-serving-cert\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.324026 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.323935    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-console-config\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.324026 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.323970    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-oauth-config\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.324026 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.323992    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-trusted-ca-bundle\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.425143 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.425099    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-serving-cert\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.425143 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.425143    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9jvr\" (UniqueName: \"kubernetes.io/projected/5131399e-939c-4528-8373-f83bc1447778-kube-api-access-d9jvr\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.425350 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.425216    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-service-ca\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.425350 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.425255    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-oauth-serving-cert\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.425350 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.425282    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-console-config\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.425350 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.425312    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-oauth-config\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.425350 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.425346    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-trusted-ca-bundle\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.426075 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.426052    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-service-ca\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.426196 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.426130    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-oauth-serving-cert\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.426196 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.426146    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-console-config\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.426754 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.426719    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-trusted-ca-bundle\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.427888 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.427867    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-oauth-config\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.428161 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.428139    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-serving-cert\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.434049 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.434020    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9jvr\" (UniqueName: \"kubernetes.io/projected/5131399e-939c-4528-8373-f83bc1447778-kube-api-access-d9jvr\") pod \"console-7c985c5888-wmxxj\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") " pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.529702 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.529664    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:35.663210 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.663176    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c985c5888-wmxxj"]
Jul 28 12:57:35.666675 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:57:35.666645    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5131399e_939c_4528_8373_f83bc1447778.slice/crio-dc71781ff2f1aa67b9e243ab175d60509adf8afb7d7d8104ccfcb96892555b39 WatchSource:0}: Error finding container dc71781ff2f1aa67b9e243ab175d60509adf8afb7d7d8104ccfcb96892555b39: Status 404 returned error can't find the container with id dc71781ff2f1aa67b9e243ab175d60509adf8afb7d7d8104ccfcb96892555b39
Jul 28 12:57:35.759898 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.759855    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c985c5888-wmxxj" event={"ID":"5131399e-939c-4528-8373-f83bc1447778","Type":"ContainerStarted","Data":"a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53"}
Jul 28 12:57:35.760104 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.759946    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c985c5888-wmxxj" event={"ID":"5131399e-939c-4528-8373-f83bc1447778","Type":"ContainerStarted","Data":"dc71781ff2f1aa67b9e243ab175d60509adf8afb7d7d8104ccfcb96892555b39"}
Jul 28 12:57:35.779452 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:35.779381    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c985c5888-wmxxj" podStartSLOduration=0.779363569 podStartE2EDuration="779.363569ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:57:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:57:35.778075185 +0000 UTC m=+236.423380485" watchObservedRunningTime="2026-07-28 12:57:35.779363569 +0000 UTC m=+236.424668860"
Jul 28 12:57:45.530146 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:45.530104    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:45.530674 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:45.530175    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:45.534973 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:45.534949    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:45.796764 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:45.796690    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 12:57:45.849726 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:57:45.849687    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fccb4cd67-ltnhd"]
Jul 28 12:58:10.868871 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:10.868761    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fccb4cd67-ltnhd" podUID="761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" containerName="console" containerID="cri-o://0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e" gracePeriod=15
Jul 28 12:58:11.114207 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.114183    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:58:11.187288 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187199    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-service-ca\") pod \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") "
Jul 28 12:58:11.187288 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187239    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-config\") pod \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") "
Jul 28 12:58:11.187506 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187341    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnvkh\" (UniqueName: \"kubernetes.io/projected/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-kube-api-access-fnvkh\") pod \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") "
Jul 28 12:58:11.187506 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187419    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-trusted-ca-bundle\") pod \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") "
Jul 28 12:58:11.187506 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187458    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-oauth-serving-cert\") pod \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") "
Jul 28 12:58:11.187681 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187514    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-serving-cert\") pod \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") "
Jul 28 12:58:11.187681 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187562    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-oauth-config\") pod \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\" (UID: \"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e\") "
Jul 28 12:58:11.187820 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187683    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-service-ca" (OuterVolumeSpecName: "service-ca") pod "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" (UID: "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:58:11.187820 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187703    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-config" (OuterVolumeSpecName: "console-config") pod "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" (UID: "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:58:11.187932 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187913    2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-service-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:58:11.187979 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187939    2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:58:11.187979 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187915    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" (UID: "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:58:11.187979 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.187932    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" (UID: "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:58:11.189554 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.189521    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" (UID: "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:58:11.189665 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.189548    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-kube-api-access-fnvkh" (OuterVolumeSpecName: "kube-api-access-fnvkh") pod "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" (UID: "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e"). InnerVolumeSpecName "kube-api-access-fnvkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:58:11.189665 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.189611    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" (UID: "761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:58:11.289197 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.289141    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnvkh\" (UniqueName: \"kubernetes.io/projected/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-kube-api-access-fnvkh\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:58:11.289197 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.289191    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-trusted-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:58:11.289197 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.289201    2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-oauth-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:58:11.289197 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.289210    2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:58:11.289482 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.289219    2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e-console-oauth-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:58:11.874559 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.874526    2573 generic.go:356] "Generic (PLEG): container finished" podID="761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" containerID="0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e" exitCode=0
Jul 28 12:58:11.874951 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.874568    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fccb4cd67-ltnhd"
Jul 28 12:58:11.874951 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.874608    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fccb4cd67-ltnhd" event={"ID":"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e","Type":"ContainerDied","Data":"0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e"}
Jul 28 12:58:11.874951 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.874660    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fccb4cd67-ltnhd" event={"ID":"761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e","Type":"ContainerDied","Data":"a2a2deab19edb3b7c5d22019a138af595ccd2e198db6f57842615a22fe52e687"}
Jul 28 12:58:11.874951 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.874683    2573 scope.go:117] "RemoveContainer" containerID="0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e"
Jul 28 12:58:11.882530 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.882510    2573 scope.go:117] "RemoveContainer" containerID="0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e"
Jul 28 12:58:11.882842 ip-10-0-132-6 kubenswrapper[2573]: E0728 12:58:11.882819    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e\": container with ID starting with 0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e not found: ID does not exist" containerID="0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e"
Jul 28 12:58:11.882926 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.882855    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e"} err="failed to get container status \"0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e\": rpc error: code = NotFound desc = could not find container \"0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e\": container with ID starting with 0fbd5ee111cbd499a63e41e35a93f8edd9b85c7fc2e87e0679b48ae6aaf01a4e not found: ID does not exist"
Jul 28 12:58:11.892977 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.892950    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fccb4cd67-ltnhd"]
Jul 28 12:58:11.896587 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:11.896561    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fccb4cd67-ltnhd"]
Jul 28 12:58:13.861767 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:13.861731    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" path="/var/lib/kubelet/pods/761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e/volumes"
Jul 28 12:58:39.689016 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:58:39.688991    2573 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 12:59:13.491996 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.491956    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"]
Jul 28 12:59:13.494121 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.492234    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" containerName="console"
Jul 28 12:59:13.494121 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.492247    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" containerName="console"
Jul 28 12:59:13.494121 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.492288    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="761fef6a-9b9f-4cfc-b2a1-926d66d8cb8e" containerName="console"
Jul 28 12:59:13.495114 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.495097    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.498255 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.498234    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8fc6j\""
Jul 28 12:59:13.498388 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.498370    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 12:59:13.499619 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.499602    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 12:59:13.505166 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.505144    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"]
Jul 28 12:59:13.544848 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.544814    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.545005 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.544862    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.545005 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.544933    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgzd\" (UniqueName: \"kubernetes.io/projected/a8d530f5-396a-41be-8b08-1994b1bdad98-kube-api-access-wfgzd\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.646000 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.645964    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgzd\" (UniqueName: \"kubernetes.io/projected/a8d530f5-396a-41be-8b08-1994b1bdad98-kube-api-access-wfgzd\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.646186 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.646008    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.646186 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.646144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.646355 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.646328    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.646481 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.646463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.657100 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.657078    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgzd\" (UniqueName: \"kubernetes.io/projected/a8d530f5-396a-41be-8b08-1994b1bdad98-kube-api-access-wfgzd\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.804643 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.804553    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:13.939062 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.939035    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"]
Jul 28 12:59:13.941594 ip-10-0-132-6 kubenswrapper[2573]: W0728 12:59:13.941565    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d530f5_396a_41be_8b08_1994b1bdad98.slice/crio-0537b2c1830b2386f91362781856bfbd192ceb9dd58aa31f90c0f06192099d47 WatchSource:0}: Error finding container 0537b2c1830b2386f91362781856bfbd192ceb9dd58aa31f90c0f06192099d47: Status 404 returned error can't find the container with id 0537b2c1830b2386f91362781856bfbd192ceb9dd58aa31f90c0f06192099d47
Jul 28 12:59:13.943573 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:13.943553    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:59:14.080961 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:14.080884    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck" event={"ID":"a8d530f5-396a-41be-8b08-1994b1bdad98","Type":"ContainerStarted","Data":"0537b2c1830b2386f91362781856bfbd192ceb9dd58aa31f90c0f06192099d47"}
Jul 28 12:59:20.105613 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:20.105578    2573 generic.go:356] "Generic (PLEG): container finished" podID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerID="7863ba2f405aaf45200822adce084088503de5c7f66ed5f22e36efb2f7b7ac45" exitCode=0
Jul 28 12:59:20.106190 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:20.105621    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck" event={"ID":"a8d530f5-396a-41be-8b08-1994b1bdad98","Type":"ContainerDied","Data":"7863ba2f405aaf45200822adce084088503de5c7f66ed5f22e36efb2f7b7ac45"}
Jul 28 12:59:25.122946 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:25.122902    2573 generic.go:356] "Generic (PLEG): container finished" podID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerID="40345262e85d0c1c62cac565f5262cb199159e49b006a78cfa5c3e7386121af6" exitCode=0
Jul 28 12:59:25.123338 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:25.122955    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck" event={"ID":"a8d530f5-396a-41be-8b08-1994b1bdad98","Type":"ContainerDied","Data":"40345262e85d0c1c62cac565f5262cb199159e49b006a78cfa5c3e7386121af6"}
Jul 28 12:59:33.151605 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:33.151565    2573 generic.go:356] "Generic (PLEG): container finished" podID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerID="0f5d0d26ef659d79b55450e4dd2d6cab9740ec0e1e9bdbde5d114c6561df098b" exitCode=0
Jul 28 12:59:33.152034 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:33.151655    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck" event={"ID":"a8d530f5-396a-41be-8b08-1994b1bdad98","Type":"ContainerDied","Data":"0f5d0d26ef659d79b55450e4dd2d6cab9740ec0e1e9bdbde5d114c6561df098b"}
Jul 28 12:59:34.279989 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.279964    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:34.393235 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.393192    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-util\") pod \"a8d530f5-396a-41be-8b08-1994b1bdad98\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") "
Jul 28 12:59:34.393439 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.393256    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfgzd\" (UniqueName: \"kubernetes.io/projected/a8d530f5-396a-41be-8b08-1994b1bdad98-kube-api-access-wfgzd\") pod \"a8d530f5-396a-41be-8b08-1994b1bdad98\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") "
Jul 28 12:59:34.393439 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.393281    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-bundle\") pod \"a8d530f5-396a-41be-8b08-1994b1bdad98\" (UID: \"a8d530f5-396a-41be-8b08-1994b1bdad98\") "
Jul 28 12:59:34.394109 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.394071    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-bundle" (OuterVolumeSpecName: "bundle") pod "a8d530f5-396a-41be-8b08-1994b1bdad98" (UID: "a8d530f5-396a-41be-8b08-1994b1bdad98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:59:34.395599 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.395573    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d530f5-396a-41be-8b08-1994b1bdad98-kube-api-access-wfgzd" (OuterVolumeSpecName: "kube-api-access-wfgzd") pod "a8d530f5-396a-41be-8b08-1994b1bdad98" (UID: "a8d530f5-396a-41be-8b08-1994b1bdad98"). InnerVolumeSpecName "kube-api-access-wfgzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:59:34.398278 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.398252    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-util" (OuterVolumeSpecName: "util") pod "a8d530f5-396a-41be-8b08-1994b1bdad98" (UID: "a8d530f5-396a-41be-8b08-1994b1bdad98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:59:34.494779 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.494740    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfgzd\" (UniqueName: \"kubernetes.io/projected/a8d530f5-396a-41be-8b08-1994b1bdad98-kube-api-access-wfgzd\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:59:34.494779 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.494778    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:59:34.494966 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:34.494788    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d530f5-396a-41be-8b08-1994b1bdad98-util\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 12:59:35.160602 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:35.160574    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck"
Jul 28 12:59:35.160602 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:35.160582    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5b4tck" event={"ID":"a8d530f5-396a-41be-8b08-1994b1bdad98","Type":"ContainerDied","Data":"0537b2c1830b2386f91362781856bfbd192ceb9dd58aa31f90c0f06192099d47"}
Jul 28 12:59:35.160860 ip-10-0-132-6 kubenswrapper[2573]: I0728 12:59:35.160624    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0537b2c1830b2386f91362781856bfbd192ceb9dd58aa31f90c0f06192099d47"
Jul 28 13:01:06.239724 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239634    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-52q9v"]
Jul 28 13:01:06.240541 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239867    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerName="extract"
Jul 28 13:01:06.240541 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239878    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerName="extract"
Jul 28 13:01:06.240541 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239889    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerName="util"
Jul 28 13:01:06.240541 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239896    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerName="util"
Jul 28 13:01:06.240541 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239906    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerName="pull"
Jul 28 13:01:06.240541 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239912    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerName="pull"
Jul 28 13:01:06.240541 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.239949    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d530f5-396a-41be-8b08-1994b1bdad98" containerName="extract"
Jul 28 13:01:06.241606 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.241589    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:06.244811 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.244768    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 28 13:01:06.244811 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.244791    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 28 13:01:06.244985 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.244842    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 28 13:01:06.246125 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.246107    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-nbxsg\""
Jul 28 13:01:06.261093 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.261053    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-52q9v"]
Jul 28 13:01:06.372278 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.372237    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34d6af1c-2eca-4dad-909d-99b06669a84b-cert\") pod \"llmisvc-controller-manager-84fc959947-52q9v\" (UID: \"34d6af1c-2eca-4dad-909d-99b06669a84b\") " pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:06.372278 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.372282    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snznk\" (UniqueName: \"kubernetes.io/projected/34d6af1c-2eca-4dad-909d-99b06669a84b-kube-api-access-snznk\") pod \"llmisvc-controller-manager-84fc959947-52q9v\" (UID: \"34d6af1c-2eca-4dad-909d-99b06669a84b\") " pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:06.473342 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.473302    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34d6af1c-2eca-4dad-909d-99b06669a84b-cert\") pod \"llmisvc-controller-manager-84fc959947-52q9v\" (UID: \"34d6af1c-2eca-4dad-909d-99b06669a84b\") " pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:06.473523 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.473348    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snznk\" (UniqueName: \"kubernetes.io/projected/34d6af1c-2eca-4dad-909d-99b06669a84b-kube-api-access-snznk\") pod \"llmisvc-controller-manager-84fc959947-52q9v\" (UID: \"34d6af1c-2eca-4dad-909d-99b06669a84b\") " pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:06.473523 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:01:06.473465    2573 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 13:01:06.473523 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:01:06.473521    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34d6af1c-2eca-4dad-909d-99b06669a84b-cert podName:34d6af1c-2eca-4dad-909d-99b06669a84b nodeName:}" failed. No retries permitted until 2026-07-28 13:01:06.973505905 +0000 UTC m=+447.618811184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/34d6af1c-2eca-4dad-909d-99b06669a84b-cert") pod "llmisvc-controller-manager-84fc959947-52q9v" (UID: "34d6af1c-2eca-4dad-909d-99b06669a84b") : secret "llmisvc-webhook-server-cert" not found
Jul 28 13:01:06.483127 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.483099    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snznk\" (UniqueName: \"kubernetes.io/projected/34d6af1c-2eca-4dad-909d-99b06669a84b-kube-api-access-snznk\") pod \"llmisvc-controller-manager-84fc959947-52q9v\" (UID: \"34d6af1c-2eca-4dad-909d-99b06669a84b\") " pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:06.978381 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.978339    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34d6af1c-2eca-4dad-909d-99b06669a84b-cert\") pod \"llmisvc-controller-manager-84fc959947-52q9v\" (UID: \"34d6af1c-2eca-4dad-909d-99b06669a84b\") " pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:06.980793 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:06.980772    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34d6af1c-2eca-4dad-909d-99b06669a84b-cert\") pod \"llmisvc-controller-manager-84fc959947-52q9v\" (UID: \"34d6af1c-2eca-4dad-909d-99b06669a84b\") " pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:07.153599 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:07.153551    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:07.282390 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:07.282351    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-52q9v"]
Jul 28 13:01:07.285154 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:01:07.285119    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34d6af1c_2eca_4dad_909d_99b06669a84b.slice/crio-e6f54a50c344cb87945e19a6b7862c01846854aa42730fb29418807859a0bbe9 WatchSource:0}: Error finding container e6f54a50c344cb87945e19a6b7862c01846854aa42730fb29418807859a0bbe9: Status 404 returned error can't find the container with id e6f54a50c344cb87945e19a6b7862c01846854aa42730fb29418807859a0bbe9
Jul 28 13:01:07.466115 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:07.466081    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-52q9v" event={"ID":"34d6af1c-2eca-4dad-909d-99b06669a84b","Type":"ContainerStarted","Data":"e6f54a50c344cb87945e19a6b7862c01846854aa42730fb29418807859a0bbe9"}
Jul 28 13:01:10.481910 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:10.481873    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-52q9v" event={"ID":"34d6af1c-2eca-4dad-909d-99b06669a84b","Type":"ContainerStarted","Data":"21f8d6a3ef6100cb3e5dbfd3530b0617fb65ddaac1de538bd41f9f2e76dad01a"}
Jul 28 13:01:10.482312 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:10.481999    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:10.501857 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:10.501800    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-84fc959947-52q9v" podStartSLOduration=1.8702936810000002 podStartE2EDuration="4.501783808s" totalImagesPullingTime="2.631490127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:01:10.500665131 +0000 UTC m=+451.145970429" watchObservedRunningTime="2026-07-28 13:01:10.501783808 +0000 UTC m=+451.147089105"
Jul 28 13:01:21.213601 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.213565    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75b4949448-4d55d"]
Jul 28 13:01:21.215749 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.215732    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.228021 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.227991    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b4949448-4d55d"]
Jul 28 13:01:21.294686 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.294644    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-console-config\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.294686 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.294686    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-oauth-serving-cert\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.294903 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.294771    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbcdw\" (UniqueName: \"kubernetes.io/projected/d490961f-108d-45b4-a2bb-175ef60f65d7-kube-api-access-kbcdw\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.294903 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.294814    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d490961f-108d-45b4-a2bb-175ef60f65d7-console-oauth-config\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.294903 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.294848    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-service-ca\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.294903 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.294895    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d490961f-108d-45b4-a2bb-175ef60f65d7-console-serving-cert\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.295029 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.294919    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-trusted-ca-bundle\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396015 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.395975    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-service-ca\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396015 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d490961f-108d-45b4-a2bb-175ef60f65d7-console-serving-cert\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396252 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-trusted-ca-bundle\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396252 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396069    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-console-config\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396252 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396218    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-oauth-serving-cert\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396445 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396291    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbcdw\" (UniqueName: \"kubernetes.io/projected/d490961f-108d-45b4-a2bb-175ef60f65d7-kube-api-access-kbcdw\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396445 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396336    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d490961f-108d-45b4-a2bb-175ef60f65d7-console-oauth-config\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396991 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396964    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-console-config\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.396991 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396983    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-service-ca\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.397177 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.396992    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-oauth-serving-cert\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.397177 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.397047    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d490961f-108d-45b4-a2bb-175ef60f65d7-trusted-ca-bundle\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.398678 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.398655    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d490961f-108d-45b4-a2bb-175ef60f65d7-console-oauth-config\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.398812 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.398795    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d490961f-108d-45b4-a2bb-175ef60f65d7-console-serving-cert\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.406752 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.406728    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbcdw\" (UniqueName: \"kubernetes.io/projected/d490961f-108d-45b4-a2bb-175ef60f65d7-kube-api-access-kbcdw\") pod \"console-75b4949448-4d55d\" (UID: \"d490961f-108d-45b4-a2bb-175ef60f65d7\") " pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.526041 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.525999    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:21.659552 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:21.659525    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b4949448-4d55d"]
Jul 28 13:01:21.661868 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:01:21.661837    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd490961f_108d_45b4_a2bb_175ef60f65d7.slice/crio-432d83182ce58e1ccddfbcc2eaf84ce5040d329f9d3b3ecbdc2e2a200ac47e26 WatchSource:0}: Error finding container 432d83182ce58e1ccddfbcc2eaf84ce5040d329f9d3b3ecbdc2e2a200ac47e26: Status 404 returned error can't find the container with id 432d83182ce58e1ccddfbcc2eaf84ce5040d329f9d3b3ecbdc2e2a200ac47e26
Jul 28 13:01:22.523342 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:22.523302    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b4949448-4d55d" event={"ID":"d490961f-108d-45b4-a2bb-175ef60f65d7","Type":"ContainerStarted","Data":"4957e3f9dcb7e43e89e25745abd4272103bdfcae33f85c9f85f8776389baa6c9"}
Jul 28 13:01:22.523740 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:22.523358    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b4949448-4d55d" event={"ID":"d490961f-108d-45b4-a2bb-175ef60f65d7","Type":"ContainerStarted","Data":"432d83182ce58e1ccddfbcc2eaf84ce5040d329f9d3b3ecbdc2e2a200ac47e26"}
Jul 28 13:01:22.544766 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:22.544714    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75b4949448-4d55d" podStartSLOduration=1.544697303 podStartE2EDuration="1.544697303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:01:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:01:22.543789227 +0000 UTC m=+463.189094524" watchObservedRunningTime="2026-07-28 13:01:22.544697303 +0000 UTC m=+463.190002672"
Jul 28 13:01:31.526679 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:31.526632    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:31.527077 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:31.526698    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:31.531439 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:31.531412    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:31.560391 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:31.560361    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75b4949448-4d55d"
Jul 28 13:01:31.612167 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:31.612100    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c985c5888-wmxxj"]
Jul 28 13:01:41.487697 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:41.487662    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-84fc959947-52q9v"
Jul 28 13:01:56.637651 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.637591    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c985c5888-wmxxj" podUID="5131399e-939c-4528-8373-f83bc1447778" containerName="console" containerID="cri-o://a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53" gracePeriod=15
Jul 28 13:01:56.874659 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.874634    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 13:01:56.974743 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.974658    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-oauth-serving-cert\") pod \"5131399e-939c-4528-8373-f83bc1447778\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") "
Jul 28 13:01:56.974743 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.974702    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-trusted-ca-bundle\") pod \"5131399e-939c-4528-8373-f83bc1447778\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") "
Jul 28 13:01:56.974743 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.974734    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9jvr\" (UniqueName: \"kubernetes.io/projected/5131399e-939c-4528-8373-f83bc1447778-kube-api-access-d9jvr\") pod \"5131399e-939c-4528-8373-f83bc1447778\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") "
Jul 28 13:01:56.975006 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.974840    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-serving-cert\") pod \"5131399e-939c-4528-8373-f83bc1447778\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") "
Jul 28 13:01:56.975006 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.974890    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-service-ca\") pod \"5131399e-939c-4528-8373-f83bc1447778\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") "
Jul 28 13:01:56.975006 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.974919    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-console-config\") pod \"5131399e-939c-4528-8373-f83bc1447778\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") "
Jul 28 13:01:56.975006 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.974943    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-oauth-config\") pod \"5131399e-939c-4528-8373-f83bc1447778\" (UID: \"5131399e-939c-4528-8373-f83bc1447778\") "
Jul 28 13:01:56.975206 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.975058    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5131399e-939c-4528-8373-f83bc1447778" (UID: "5131399e-939c-4528-8373-f83bc1447778"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:01:56.975260 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.975220    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5131399e-939c-4528-8373-f83bc1447778" (UID: "5131399e-939c-4528-8373-f83bc1447778"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:01:56.975348 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.975328    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-trusted-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:01:56.975418 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.975357    2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-oauth-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:01:56.975418 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.975325    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-console-config" (OuterVolumeSpecName: "console-config") pod "5131399e-939c-4528-8373-f83bc1447778" (UID: "5131399e-939c-4528-8373-f83bc1447778"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:01:56.975418 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.975322    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-service-ca" (OuterVolumeSpecName: "service-ca") pod "5131399e-939c-4528-8373-f83bc1447778" (UID: "5131399e-939c-4528-8373-f83bc1447778"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:01:56.977079 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.977049    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5131399e-939c-4528-8373-f83bc1447778" (UID: "5131399e-939c-4528-8373-f83bc1447778"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:01:56.977202 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.977092    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5131399e-939c-4528-8373-f83bc1447778" (UID: "5131399e-939c-4528-8373-f83bc1447778"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:01:56.977202 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:56.977101    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5131399e-939c-4528-8373-f83bc1447778-kube-api-access-d9jvr" (OuterVolumeSpecName: "kube-api-access-d9jvr") pod "5131399e-939c-4528-8373-f83bc1447778" (UID: "5131399e-939c-4528-8373-f83bc1447778"). InnerVolumeSpecName "kube-api-access-d9jvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:01:57.075871 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.075832    2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-service-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:01:57.075871 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.075862    2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5131399e-939c-4528-8373-f83bc1447778-console-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:01:57.075871 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.075876    2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-oauth-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:01:57.076085 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.075888    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9jvr\" (UniqueName: \"kubernetes.io/projected/5131399e-939c-4528-8373-f83bc1447778-kube-api-access-d9jvr\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:01:57.076085 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.075901    2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5131399e-939c-4528-8373-f83bc1447778-console-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:01:57.651590 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.651555    2573 generic.go:356] "Generic (PLEG): container finished" podID="5131399e-939c-4528-8373-f83bc1447778" containerID="a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53" exitCode=0
Jul 28 13:01:57.651974 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.651597    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c985c5888-wmxxj"
Jul 28 13:01:57.651974 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.651642    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c985c5888-wmxxj" event={"ID":"5131399e-939c-4528-8373-f83bc1447778","Type":"ContainerDied","Data":"a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53"}
Jul 28 13:01:57.651974 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.651690    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c985c5888-wmxxj" event={"ID":"5131399e-939c-4528-8373-f83bc1447778","Type":"ContainerDied","Data":"dc71781ff2f1aa67b9e243ab175d60509adf8afb7d7d8104ccfcb96892555b39"}
Jul 28 13:01:57.651974 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.651710    2573 scope.go:117] "RemoveContainer" containerID="a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53"
Jul 28 13:01:57.660567 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.660550    2573 scope.go:117] "RemoveContainer" containerID="a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53"
Jul 28 13:01:57.660820 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:01:57.660803    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53\": container with ID starting with a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53 not found: ID does not exist" containerID="a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53"
Jul 28 13:01:57.660857 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.660829    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53"} err="failed to get container status \"a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53\": rpc error: code = NotFound desc = could not find container \"a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53\": container with ID starting with a1350c7493247815d759adb76ca912bf0bfd2ec90ccb3a77e6fdd762c8af0d53 not found: ID does not exist"
Jul 28 13:01:57.673092 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.673071    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c985c5888-wmxxj"]
Jul 28 13:01:57.675374 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.675350    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c985c5888-wmxxj"]
Jul 28 13:01:57.862180 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:01:57.862139    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5131399e-939c-4528-8373-f83bc1447778" path="/var/lib/kubelet/pods/5131399e-939c-4528-8373-f83bc1447778/volumes"
Jul 28 13:04:15.836456 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.836416    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"]
Jul 28 13:04:15.838799 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.836789    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5131399e-939c-4528-8373-f83bc1447778" containerName="console"
Jul 28 13:04:15.838799 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.836808    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5131399e-939c-4528-8373-f83bc1447778" containerName="console"
Jul 28 13:04:15.838799 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.836874    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5131399e-939c-4528-8373-f83bc1447778" containerName="console"
Jul 28 13:04:15.839657 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.839639    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:15.843930 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.843894    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 28 13:04:15.844066 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.843932    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 28 13:04:15.844066 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.843943    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Jul 28 13:04:15.844066 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.843961    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 28 13:04:15.844066 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.843904    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-pkltr\""
Jul 28 13:04:15.844066 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.843904    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 28 13:04:15.844066 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.843894    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Jul 28 13:04:15.849571 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.849540    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"]
Jul 28 13:04:15.966012 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.965972    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:15.966012 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.966022    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:15.966238 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.966046    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:15.966238 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.966064    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kztd\" (UniqueName: \"kubernetes.io/projected/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-api-access-2kztd\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:15.966238 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:15.966124    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067078 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067240 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067119    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067240 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067169    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067240 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067201    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067240 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067221    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kztd\" (UniqueName: \"kubernetes.io/projected/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-api-access-2kztd\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067646 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067622    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067789 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067769    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.067844 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.067824    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.069570 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.069551    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.078205 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.078174    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kztd\" (UniqueName: \"kubernetes.io/projected/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-api-access-2kztd\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.151907 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.151809    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:16.290140 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.290107    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"]
Jul 28 13:04:16.293506 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:04:16.293468    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb1d2bf_9452_4d32_9c72_7fe727ec9202.slice/crio-2b4f6d44df31dea51ecc56258537f4341045180af01aec5f4d5647b5be7e2655 WatchSource:0}: Error finding container 2b4f6d44df31dea51ecc56258537f4341045180af01aec5f4d5647b5be7e2655: Status 404 returned error can't find the container with id 2b4f6d44df31dea51ecc56258537f4341045180af01aec5f4d5647b5be7e2655
Jul 28 13:04:16.295643 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:16.295623    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:04:17.123591 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:17.123558    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerStarted","Data":"2b4f6d44df31dea51ecc56258537f4341045180af01aec5f4d5647b5be7e2655"}
Jul 28 13:04:21.143768 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:21.143733    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerStarted","Data":"b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7"}
Jul 28 13:04:25.159564 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:25.159530    2573 generic.go:356] "Generic (PLEG): container finished" podID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerID="b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7" exitCode=0
Jul 28 13:04:25.159910 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:25.159573    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerDied","Data":"b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7"}
Jul 28 13:04:38.219046 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:38.218968    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerStarted","Data":"17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55"}
Jul 28 13:04:41.232369 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:41.232329    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerStarted","Data":"f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570"}
Jul 28 13:04:41.232784 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:41.232632    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:41.232784 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:41.232750    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:41.234105 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:41.234072    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:04:41.253244 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:41.253201    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podStartSLOduration=3.019151178 podStartE2EDuration="26.253189s" totalImagesPullingTime="19.338674845s" totalInitContainerRuntime="3.895362977s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:04:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:04:41.252202528 +0000 UTC m=+661.897507825" watchObservedRunningTime="2026-07-28 13:04:41.253189 +0000 UTC m=+661.898494297"
Jul 28 13:04:42.236725 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:42.236677    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:04:47.241061 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:47.241030    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:04:47.241792 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:47.241759    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:04:57.242160 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:04:57.242113    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:05:07.241729 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:07.241688    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:05:17.242327 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:17.242273    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:05:27.241674 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:27.241631    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:05:35.185463 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.185362    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"]
Jul 28 13:05:35.187464 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.187448    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:35.190975 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.190953    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-aee94-serving-cert\""
Jul 28 13:05:35.191093 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.190954    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-aee94-kube-rbac-proxy-sar-config\""
Jul 28 13:05:35.201501 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.201474    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"]
Jul 28 13:05:35.276321 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.276283    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls\") pod \"switch-graph-aee94-5cd98b64b4-cglpz\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") " pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:35.276511 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.276332    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-openshift-service-ca-bundle\") pod \"switch-graph-aee94-5cd98b64b4-cglpz\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") " pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:35.377214 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.377172    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls\") pod \"switch-graph-aee94-5cd98b64b4-cglpz\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") " pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:35.377346 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.377225    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-openshift-service-ca-bundle\") pod \"switch-graph-aee94-5cd98b64b4-cglpz\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") " pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:35.377346 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:05:35.377325    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-aee94-serving-cert: secret "switch-graph-aee94-serving-cert" not found
Jul 28 13:05:35.377449 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:05:35.377427    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls podName:6d21c0a2-690c-4f1a-aae6-9a9edeb550ce nodeName:}" failed. No retries permitted until 2026-07-28 13:05:35.8773853 +0000 UTC m=+716.522690576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls") pod "switch-graph-aee94-5cd98b64b4-cglpz" (UID: "6d21c0a2-690c-4f1a-aae6-9a9edeb550ce") : secret "switch-graph-aee94-serving-cert" not found
Jul 28 13:05:35.377859 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.377841    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-openshift-service-ca-bundle\") pod \"switch-graph-aee94-5cd98b64b4-cglpz\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") " pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:35.883220 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.883174    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls\") pod \"switch-graph-aee94-5cd98b64b4-cglpz\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") " pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:35.885772 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:35.885748    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls\") pod \"switch-graph-aee94-5cd98b64b4-cglpz\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") " pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:36.096839 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:36.096794    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:36.229598 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:36.229568    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"]
Jul 28 13:05:36.232364 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:05:36.232338    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d21c0a2_690c_4f1a_aae6_9a9edeb550ce.slice/crio-bf1e803dc329e1a3d041cea45162f65186f8b4cd418ce96830e2ed70ddbff615 WatchSource:0}: Error finding container bf1e803dc329e1a3d041cea45162f65186f8b4cd418ce96830e2ed70ddbff615: Status 404 returned error can't find the container with id bf1e803dc329e1a3d041cea45162f65186f8b4cd418ce96830e2ed70ddbff615
Jul 28 13:05:36.428558 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:36.428475    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" event={"ID":"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce","Type":"ContainerStarted","Data":"bf1e803dc329e1a3d041cea45162f65186f8b4cd418ce96830e2ed70ddbff615"}
Jul 28 13:05:37.242411 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:37.242355    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:05:39.441645 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:39.441546    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" event={"ID":"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce","Type":"ContainerStarted","Data":"c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16"}
Jul 28 13:05:39.442057 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:39.441666    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:39.459433 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:39.459365    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podStartSLOduration=1.576635146 podStartE2EDuration="4.459348706s" totalImagesPullingTime="2.88271356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:05:39.458970008 +0000 UTC m=+720.104275307" watchObservedRunningTime="2026-07-28 13:05:39.459348706 +0000 UTC m=+720.104654046"
Jul 28 13:05:45.452021 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:45.451986    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:05:47.241606 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:47.241571    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:05:49.379370 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:49.379335    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"]
Jul 28 13:05:49.379744 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:49.379576    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94" containerID="cri-o://c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16" gracePeriod=30
Jul 28 13:05:50.450144 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:50.450108    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:05:55.449973 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:55.449931    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:05:57.242287 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:05:57.242258    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:06:00.449682 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:00.449636    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:00.450138 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:00.449775    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:06:05.449301 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:05.449255    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:10.449864 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:10.449825    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:15.238163 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.238123    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"]
Jul 28 13:06:15.244127 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.244099    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:15.247271 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.247249    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Jul 28 13:06:15.247362 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.247250    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Jul 28 13:06:15.254825 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.254799    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"]
Jul 28 13:06:15.285290 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.285254    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96378e72-6ba9-40dc-989a-d8b9f4ca4982-openshift-service-ca-bundle\") pod \"model-chainer-b84f4c8c7-bkw6s\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") " pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:15.285495 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.285300    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls\") pod \"model-chainer-b84f4c8c7-bkw6s\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") " pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:15.386632 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.386583    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96378e72-6ba9-40dc-989a-d8b9f4ca4982-openshift-service-ca-bundle\") pod \"model-chainer-b84f4c8c7-bkw6s\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") " pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:15.386804 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.386659    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls\") pod \"model-chainer-b84f4c8c7-bkw6s\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") " pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:15.386847 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:15.386829    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found
Jul 28 13:06:15.386911 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:15.386900    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls podName:96378e72-6ba9-40dc-989a-d8b9f4ca4982 nodeName:}" failed. No retries permitted until 2026-07-28 13:06:15.886878626 +0000 UTC m=+756.532183902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls") pod "model-chainer-b84f4c8c7-bkw6s" (UID: "96378e72-6ba9-40dc-989a-d8b9f4ca4982") : secret "model-chainer-serving-cert" not found
Jul 28 13:06:15.387220 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.387202    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96378e72-6ba9-40dc-989a-d8b9f4ca4982-openshift-service-ca-bundle\") pod \"model-chainer-b84f4c8c7-bkw6s\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") " pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:15.450845 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.450795    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:15.890475 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.890435    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls\") pod \"model-chainer-b84f4c8c7-bkw6s\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") " pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:15.892886 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:15.892864    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls\") pod \"model-chainer-b84f4c8c7-bkw6s\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") " pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:16.155343 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:16.155245    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:16.292791 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:16.292716    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"]
Jul 28 13:06:16.295380 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:06:16.295352    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96378e72_6ba9_40dc_989a_d8b9f4ca4982.slice/crio-96fa5e6fc7ab6a15c72df4f8e8cab4fe7785e5c81031a560e3a7692aa990cf44 WatchSource:0}: Error finding container 96fa5e6fc7ab6a15c72df4f8e8cab4fe7785e5c81031a560e3a7692aa990cf44: Status 404 returned error can't find the container with id 96fa5e6fc7ab6a15c72df4f8e8cab4fe7785e5c81031a560e3a7692aa990cf44
Jul 28 13:06:16.576774 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:16.576738    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" event={"ID":"96378e72-6ba9-40dc-989a-d8b9f4ca4982","Type":"ContainerStarted","Data":"2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc"}
Jul 28 13:06:16.576972 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:16.576796    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" event={"ID":"96378e72-6ba9-40dc-989a-d8b9f4ca4982","Type":"ContainerStarted","Data":"96fa5e6fc7ab6a15c72df4f8e8cab4fe7785e5c81031a560e3a7692aa990cf44"}
Jul 28 13:06:16.576972 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:16.576850    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:16.594658 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:16.594577    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podStartSLOduration=1.59455887 podStartE2EDuration="1.59455887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:06:16.59325514 +0000 UTC m=+757.238560438" watchObservedRunningTime="2026-07-28 13:06:16.59455887 +0000 UTC m=+757.239864230"
Jul 28 13:06:19.573714 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.573687    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:06:19.588469 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.588432    2573 generic.go:356] "Generic (PLEG): container finished" podID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerID="c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16" exitCode=137
Jul 28 13:06:19.588626 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.588483    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"
Jul 28 13:06:19.588626 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.588525    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" event={"ID":"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce","Type":"ContainerDied","Data":"c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16"}
Jul 28 13:06:19.588626 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.588582    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz" event={"ID":"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce","Type":"ContainerDied","Data":"bf1e803dc329e1a3d041cea45162f65186f8b4cd418ce96830e2ed70ddbff615"}
Jul 28 13:06:19.588626 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.588606    2573 scope.go:117] "RemoveContainer" containerID="c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16"
Jul 28 13:06:19.597621 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.597596    2573 scope.go:117] "RemoveContainer" containerID="c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16"
Jul 28 13:06:19.597947 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:19.597926    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16\": container with ID starting with c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16 not found: ID does not exist" containerID="c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16"
Jul 28 13:06:19.598009 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.597956    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16"} err="failed to get container status \"c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16\": rpc error: code = NotFound desc = could not find container \"c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16\": container with ID starting with c9638e32d403bd9e788ff488d7ead6c95c0967904c13eff87d80a0667d571b16 not found: ID does not exist"
Jul 28 13:06:19.617596 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.617561    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-openshift-service-ca-bundle\") pod \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") "
Jul 28 13:06:19.617756 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.617608    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls\") pod \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\" (UID: \"6d21c0a2-690c-4f1a-aae6-9a9edeb550ce\") "
Jul 28 13:06:19.617923 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.617900    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" (UID: "6d21c0a2-690c-4f1a-aae6-9a9edeb550ce"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:06:19.619835 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.619814    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" (UID: "6d21c0a2-690c-4f1a-aae6-9a9edeb550ce"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:06:19.718810 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.718695    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:19.718810 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.718741    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:19.904869 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.904836    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"]
Jul 28 13:06:19.908642 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:19.908615    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-aee94-5cd98b64b4-cglpz"]
Jul 28 13:06:21.862887 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:21.862853    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" path="/var/lib/kubelet/pods/6d21c0a2-690c-4f1a-aae6-9a9edeb550ce/volumes"
Jul 28 13:06:22.586048 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:22.586018    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:25.346204 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:25.346172    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"]
Jul 28 13:06:25.346606 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:25.346351    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer" containerID="cri-o://2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc" gracePeriod=30
Jul 28 13:06:25.476280 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:25.476246    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"]
Jul 28 13:06:25.476586 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:25.476556    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" containerID="cri-o://17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55" gracePeriod=30
Jul 28 13:06:25.476664 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:25.476595    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kube-rbac-proxy" containerID="cri-o://f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570" gracePeriod=30
Jul 28 13:06:25.612208 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:25.612127    2573 generic.go:356] "Generic (PLEG): container finished" podID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerID="f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570" exitCode=2
Jul 28 13:06:25.612208 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:25.612191    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerDied","Data":"f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570"}
Jul 28 13:06:27.237456 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:27.237372    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 28 13:06:27.241891 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:27.241848    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 13:06:27.584156 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:27.584069    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:30.329054 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.329025    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:06:30.396843 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.396741    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-proxy-tls\") pod \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") "
Jul 28 13:06:30.397008 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.396845    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-cabundle-cert\") pod \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") "
Jul 28 13:06:30.397008 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.396885    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kserve-provision-location\") pod \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") "
Jul 28 13:06:30.397008 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.396925    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-rbac-proxy-sar-config\") pod \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") "
Jul 28 13:06:30.397008 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.396954    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kztd\" (UniqueName: \"kubernetes.io/projected/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-api-access-2kztd\") pod \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\" (UID: \"ceb1d2bf-9452-4d32-9c72-7fe727ec9202\") "
Jul 28 13:06:30.397293 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.397259    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ceb1d2bf-9452-4d32-9c72-7fe727ec9202" (UID: "ceb1d2bf-9452-4d32-9c72-7fe727ec9202"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:06:30.397293 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.397274    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ceb1d2bf-9452-4d32-9c72-7fe727ec9202" (UID: "ceb1d2bf-9452-4d32-9c72-7fe727ec9202"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:06:30.397387 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.397271    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ceb1d2bf-9452-4d32-9c72-7fe727ec9202" (UID: "ceb1d2bf-9452-4d32-9c72-7fe727ec9202"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:06:30.399056 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.399026    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ceb1d2bf-9452-4d32-9c72-7fe727ec9202" (UID: "ceb1d2bf-9452-4d32-9c72-7fe727ec9202"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:06:30.399056 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.399044    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-api-access-2kztd" (OuterVolumeSpecName: "kube-api-access-2kztd") pod "ceb1d2bf-9452-4d32-9c72-7fe727ec9202" (UID: "ceb1d2bf-9452-4d32-9c72-7fe727ec9202"). InnerVolumeSpecName "kube-api-access-2kztd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:06:30.497834 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.497790    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kztd\" (UniqueName: \"kubernetes.io/projected/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-api-access-2kztd\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:30.497834 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.497825    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:30.497834 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.497836    2573 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-cabundle-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:30.497834 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.497844    2573 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kserve-provision-location\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:30.498084 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.497853    2573 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ceb1d2bf-9452-4d32-9c72-7fe727ec9202-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:30.634147 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.634113    2573 generic.go:356] "Generic (PLEG): container finished" podID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerID="17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55" exitCode=0
Jul 28 13:06:30.634315 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.634167    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"
Jul 28 13:06:30.634315 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.634197    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerDied","Data":"17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55"}
Jul 28 13:06:30.634315 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.634258    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w" event={"ID":"ceb1d2bf-9452-4d32-9c72-7fe727ec9202","Type":"ContainerDied","Data":"2b4f6d44df31dea51ecc56258537f4341045180af01aec5f4d5647b5be7e2655"}
Jul 28 13:06:30.634315 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.634285    2573 scope.go:117] "RemoveContainer" containerID="f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570"
Jul 28 13:06:30.642348 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.642331    2573 scope.go:117] "RemoveContainer" containerID="17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55"
Jul 28 13:06:30.650014 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.649994    2573 scope.go:117] "RemoveContainer" containerID="b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7"
Jul 28 13:06:30.657584 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.657558    2573 scope.go:117] "RemoveContainer" containerID="f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570"
Jul 28 13:06:30.658025 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:30.657998    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570\": container with ID starting with f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570 not found: ID does not exist" containerID="f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570"
Jul 28 13:06:30.658186 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.658037    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570"} err="failed to get container status \"f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570\": rpc error: code = NotFound desc = could not find container \"f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570\": container with ID starting with f82aa4eaecc45e671a8c1db7168a06276d6c8b824d149e988694bf46993ee570 not found: ID does not exist"
Jul 28 13:06:30.658186 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.658062    2573 scope.go:117] "RemoveContainer" containerID="17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55"
Jul 28 13:06:30.658703 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:30.658390    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55\": container with ID starting with 17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55 not found: ID does not exist" containerID="17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55"
Jul 28 13:06:30.658796 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.658713    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55"} err="failed to get container status \"17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55\": rpc error: code = NotFound desc = could not find container \"17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55\": container with ID starting with 17dc2cf4c4ad10c54110396dffa0c0286e9afd685e61c6ba6847c0b98acd0e55 not found: ID does not exist"
Jul 28 13:06:30.658796 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.658733    2573 scope.go:117] "RemoveContainer" containerID="b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7"
Jul 28 13:06:30.659007 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:30.658987    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7\": container with ID starting with b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7 not found: ID does not exist" containerID="b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7"
Jul 28 13:06:30.659068 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.659011    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7"} err="failed to get container status \"b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7\": rpc error: code = NotFound desc = could not find container \"b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7\": container with ID starting with b02f21d7fc81e9f81e0a942ca2ea26285607d114a3f7e2703c6abb02db73b9b7 not found: ID does not exist"
Jul 28 13:06:30.659714 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.659694    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"]
Jul 28 13:06:30.665889 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:30.665863    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-wlh2w"]
Jul 28 13:06:31.862169 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:31.862135    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" path="/var/lib/kubelet/pods/ceb1d2bf-9452-4d32-9c72-7fe727ec9202/volumes"
Jul 28 13:06:32.584267 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:32.584227    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:37.584504 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:37.584453    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:37.584964 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:37.584589    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:42.583863 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:42.583814    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:47.584043 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:47.583998    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:49.635455 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635424    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"]
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635676    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635686    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635700    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635706    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635716    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="storage-initializer"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635722    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="storage-initializer"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635729    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kube-rbac-proxy"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635735    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kube-rbac-proxy"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635771    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d21c0a2-690c-4f1a-aae6-9a9edeb550ce" containerName="switch-graph-aee94"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635779    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kube-rbac-proxy"
Jul 28 13:06:49.635842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.635785    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb1d2bf-9452-4d32-9c72-7fe727ec9202" containerName="kserve-container"
Jul 28 13:06:49.639759 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.639733    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:49.642776 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.642755    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-c21f8-serving-cert\""
Jul 28 13:06:49.643371 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.643355    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-c21f8-kube-rbac-proxy-sar-config\""
Jul 28 13:06:49.649812 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.649790    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"]
Jul 28 13:06:49.733141 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.733104    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264fd1a-4ed5-4726-99d3-7fc6299caa49-openshift-service-ca-bundle\") pod \"switch-graph-c21f8-7885d4d46c-z622w\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") " pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:49.733141 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.733144    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls\") pod \"switch-graph-c21f8-7885d4d46c-z622w\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") " pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:49.834481 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.834440    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264fd1a-4ed5-4726-99d3-7fc6299caa49-openshift-service-ca-bundle\") pod \"switch-graph-c21f8-7885d4d46c-z622w\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") " pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:49.834481 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.834482    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls\") pod \"switch-graph-c21f8-7885d4d46c-z622w\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") " pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:49.834750 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:49.834649    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-c21f8-serving-cert: secret "switch-graph-c21f8-serving-cert" not found
Jul 28 13:06:49.834750 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:49.834735    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls podName:b264fd1a-4ed5-4726-99d3-7fc6299caa49 nodeName:}" failed. No retries permitted until 2026-07-28 13:06:50.33471546 +0000 UTC m=+790.980020736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls") pod "switch-graph-c21f8-7885d4d46c-z622w" (UID: "b264fd1a-4ed5-4726-99d3-7fc6299caa49") : secret "switch-graph-c21f8-serving-cert" not found
Jul 28 13:06:49.835186 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:49.835159    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264fd1a-4ed5-4726-99d3-7fc6299caa49-openshift-service-ca-bundle\") pod \"switch-graph-c21f8-7885d4d46c-z622w\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") " pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:50.339334 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:50.339276    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls\") pod \"switch-graph-c21f8-7885d4d46c-z622w\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") " pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:50.341721 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:50.341687    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls\") pod \"switch-graph-c21f8-7885d4d46c-z622w\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") " pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:50.549280 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:50.549231    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:50.678687 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:50.678658    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"]
Jul 28 13:06:50.681027 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:06:50.680995    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb264fd1a_4ed5_4726_99d3_7fc6299caa49.slice/crio-ee44eac4ab1534f4795b0bdb2119877cd52b0c05e86d9194e87127ff97730374 WatchSource:0}: Error finding container ee44eac4ab1534f4795b0bdb2119877cd52b0c05e86d9194e87127ff97730374: Status 404 returned error can't find the container with id ee44eac4ab1534f4795b0bdb2119877cd52b0c05e86d9194e87127ff97730374
Jul 28 13:06:50.704412 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:50.704363    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" event={"ID":"b264fd1a-4ed5-4726-99d3-7fc6299caa49","Type":"ContainerStarted","Data":"ee44eac4ab1534f4795b0bdb2119877cd52b0c05e86d9194e87127ff97730374"}
Jul 28 13:06:51.708857 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:51.708822    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" event={"ID":"b264fd1a-4ed5-4726-99d3-7fc6299caa49","Type":"ContainerStarted","Data":"79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60"}
Jul 28 13:06:51.709375 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:51.708887    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:06:51.727433 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:51.727367    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podStartSLOduration=2.7273522310000002 podStartE2EDuration="2.727352231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:06:51.725336813 +0000 UTC m=+792.370642113" watchObservedRunningTime="2026-07-28 13:06:51.727352231 +0000 UTC m=+792.372657529"
Jul 28 13:06:52.584100 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:52.584053    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:06:55.493124 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.493096    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:55.579615 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.579576    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96378e72-6ba9-40dc-989a-d8b9f4ca4982-openshift-service-ca-bundle\") pod \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") "
Jul 28 13:06:55.579615 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.579623    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls\") pod \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\" (UID: \"96378e72-6ba9-40dc-989a-d8b9f4ca4982\") "
Jul 28 13:06:55.579986 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.579958    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96378e72-6ba9-40dc-989a-d8b9f4ca4982-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "96378e72-6ba9-40dc-989a-d8b9f4ca4982" (UID: "96378e72-6ba9-40dc-989a-d8b9f4ca4982"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:06:55.581718 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.581689    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "96378e72-6ba9-40dc-989a-d8b9f4ca4982" (UID: "96378e72-6ba9-40dc-989a-d8b9f4ca4982"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:06:55.680628 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.680539    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96378e72-6ba9-40dc-989a-d8b9f4ca4982-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:55.680628 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.680583    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96378e72-6ba9-40dc-989a-d8b9f4ca4982-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:06:55.722372 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.722340    2573 generic.go:356] "Generic (PLEG): container finished" podID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerID="2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc" exitCode=0
Jul 28 13:06:55.722532 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.722380    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"
Jul 28 13:06:55.722532 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.722434    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" event={"ID":"96378e72-6ba9-40dc-989a-d8b9f4ca4982","Type":"ContainerDied","Data":"2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc"}
Jul 28 13:06:55.722532 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.722486    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s" event={"ID":"96378e72-6ba9-40dc-989a-d8b9f4ca4982","Type":"ContainerDied","Data":"96fa5e6fc7ab6a15c72df4f8e8cab4fe7785e5c81031a560e3a7692aa990cf44"}
Jul 28 13:06:55.722532 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.722512    2573 scope.go:117] "RemoveContainer" containerID="2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc"
Jul 28 13:06:55.730671 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.730651    2573 scope.go:117] "RemoveContainer" containerID="2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc"
Jul 28 13:06:55.730958 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:06:55.730937    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc\": container with ID starting with 2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc not found: ID does not exist" containerID="2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc"
Jul 28 13:06:55.731011 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.730969    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc"} err="failed to get container status \"2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc\": rpc error: code = NotFound desc = could not find container \"2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc\": container with ID starting with 2de926f3cf1fe3ff4cb38994da7b3e784ad6a90b24050a3281403fa081753dfc not found: ID does not exist"
Jul 28 13:06:55.744653 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.744626    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"]
Jul 28 13:06:55.748308 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.748286    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b84f4c8c7-bkw6s"]
Jul 28 13:06:55.862039 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:55.862005    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" path="/var/lib/kubelet/pods/96378e72-6ba9-40dc-989a-d8b9f4ca4982/volumes"
Jul 28 13:06:57.718347 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:06:57.718319    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:07:25.586556 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.586522    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"]
Jul 28 13:07:25.587020 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.586788    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer"
Jul 28 13:07:25.587020 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.586808    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer"
Jul 28 13:07:25.587020 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.586852    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="96378e72-6ba9-40dc-989a-d8b9f4ca4982" containerName="model-chainer"
Jul 28 13:07:25.589640 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.589624    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:25.593063 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.593032    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fec78-serving-cert\""
Jul 28 13:07:25.593063 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.593052    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fec78-kube-rbac-proxy-sar-config\""
Jul 28 13:07:25.598520 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.598497    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"]
Jul 28 13:07:25.609683 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.609660    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6485a5-90f6-405b-b73a-17b396cebfc2-openshift-service-ca-bundle\") pod \"sequence-graph-fec78-759665677-98xxj\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") " pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:25.609775 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.609743    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls\") pod \"sequence-graph-fec78-759665677-98xxj\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") " pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:25.710569 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.710525    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls\") pod \"sequence-graph-fec78-759665677-98xxj\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") " pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:25.710747 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.710626    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6485a5-90f6-405b-b73a-17b396cebfc2-openshift-service-ca-bundle\") pod \"sequence-graph-fec78-759665677-98xxj\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") " pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:25.710747 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:07:25.710664    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-fec78-serving-cert: secret "sequence-graph-fec78-serving-cert" not found
Jul 28 13:07:25.710747 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:07:25.710738    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls podName:7e6485a5-90f6-405b-b73a-17b396cebfc2 nodeName:}" failed. No retries permitted until 2026-07-28 13:07:26.210720549 +0000 UTC m=+826.856025825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls") pod "sequence-graph-fec78-759665677-98xxj" (UID: "7e6485a5-90f6-405b-b73a-17b396cebfc2") : secret "sequence-graph-fec78-serving-cert" not found
Jul 28 13:07:25.711384 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:25.711364    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6485a5-90f6-405b-b73a-17b396cebfc2-openshift-service-ca-bundle\") pod \"sequence-graph-fec78-759665677-98xxj\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") " pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:26.213946 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.213913    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls\") pod \"sequence-graph-fec78-759665677-98xxj\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") " pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:26.216209 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.216188    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls\") pod \"sequence-graph-fec78-759665677-98xxj\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") " pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:26.499738 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.499710    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:26.626954 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.626921    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"]
Jul 28 13:07:26.629830 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:07:26.629803    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6485a5_90f6_405b_b73a_17b396cebfc2.slice/crio-e9cb05b2fb3860b923087b26a0ec4ab07c7a1b02c59a2aaa4cff74860e755442 WatchSource:0}: Error finding container e9cb05b2fb3860b923087b26a0ec4ab07c7a1b02c59a2aaa4cff74860e755442: Status 404 returned error can't find the container with id e9cb05b2fb3860b923087b26a0ec4ab07c7a1b02c59a2aaa4cff74860e755442
Jul 28 13:07:26.838854 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.838750    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" event={"ID":"7e6485a5-90f6-405b-b73a-17b396cebfc2","Type":"ContainerStarted","Data":"1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f"}
Jul 28 13:07:26.838854 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.838821    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" event={"ID":"7e6485a5-90f6-405b-b73a-17b396cebfc2","Type":"ContainerStarted","Data":"e9cb05b2fb3860b923087b26a0ec4ab07c7a1b02c59a2aaa4cff74860e755442"}
Jul 28 13:07:26.839038 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.838869    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:07:26.863352 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:26.859472    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podStartSLOduration=1.859450544 podStartE2EDuration="1.859450544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:07:26.858038008 +0000 UTC m=+827.503343307" watchObservedRunningTime="2026-07-28 13:07:26.859450544 +0000 UTC m=+827.504755843"
Jul 28 13:07:32.849222 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:07:32.849193    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:15:04.334154 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:04.334060    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"]
Jul 28 13:15:04.336593 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:04.334300    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8" containerID="cri-o://79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60" gracePeriod=30
Jul 28 13:15:07.716917 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:07.716873    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:12.716231 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:12.716189    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:17.716324 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:17.716282    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:17.716719 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:17.716446    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:15:22.716947 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:22.716901    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:27.716907 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:27.716865    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:32.716789 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:32.716741    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:34.476081 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.476055    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:15:34.516451 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.516417    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls\") pod \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") "
Jul 28 13:15:34.518493 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.518471    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b264fd1a-4ed5-4726-99d3-7fc6299caa49" (UID: "b264fd1a-4ed5-4726-99d3-7fc6299caa49"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:15:34.528633 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.528600    2573 generic.go:356] "Generic (PLEG): container finished" podID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerID="79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60" exitCode=0
Jul 28 13:15:34.528788 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.528642    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"
Jul 28 13:15:34.528788 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.528686    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" event={"ID":"b264fd1a-4ed5-4726-99d3-7fc6299caa49","Type":"ContainerDied","Data":"79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60"}
Jul 28 13:15:34.528788 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.528736    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w" event={"ID":"b264fd1a-4ed5-4726-99d3-7fc6299caa49","Type":"ContainerDied","Data":"ee44eac4ab1534f4795b0bdb2119877cd52b0c05e86d9194e87127ff97730374"}
Jul 28 13:15:34.528788 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.528759    2573 scope.go:117] "RemoveContainer" containerID="79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60"
Jul 28 13:15:34.536922 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.536903    2573 scope.go:117] "RemoveContainer" containerID="79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60"
Jul 28 13:15:34.537209 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:15:34.537191    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60\": container with ID starting with 79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60 not found: ID does not exist" containerID="79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60"
Jul 28 13:15:34.537262 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.537218    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60"} err="failed to get container status \"79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60\": rpc error: code = NotFound desc = could not find container \"79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60\": container with ID starting with 79de3dcf8990a561297d9b038b5376c12024ba1b01bb207c1ef1ab6661a67b60 not found: ID does not exist"
Jul 28 13:15:34.616937 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.616849    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264fd1a-4ed5-4726-99d3-7fc6299caa49-openshift-service-ca-bundle\") pod \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\" (UID: \"b264fd1a-4ed5-4726-99d3-7fc6299caa49\") "
Jul 28 13:15:34.617078 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.617028    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b264fd1a-4ed5-4726-99d3-7fc6299caa49-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:15:34.617233 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.617213    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b264fd1a-4ed5-4726-99d3-7fc6299caa49-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b264fd1a-4ed5-4726-99d3-7fc6299caa49" (UID: "b264fd1a-4ed5-4726-99d3-7fc6299caa49"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:15:34.718217 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.718175    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264fd1a-4ed5-4726-99d3-7fc6299caa49-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:15:34.850575 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.850540    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"]
Jul 28 13:15:34.854810 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:34.854784    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c21f8-7885d4d46c-z622w"]
Jul 28 13:15:35.862089 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:35.862054    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" path="/var/lib/kubelet/pods/b264fd1a-4ed5-4726-99d3-7fc6299caa49/volumes"
Jul 28 13:15:40.187192 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:40.187105    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"]
Jul 28 13:15:40.187564 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:40.187355    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78" containerID="cri-o://1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f" gracePeriod=30
Jul 28 13:15:42.847573 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:42.847533    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:47.847276 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:47.847237    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:52.847411 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:52.847354    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:15:52.847820 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:52.847482    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:15:57.846934 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:15:57.846894    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:02.847562 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:02.847478    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:04.604043 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.604006    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"]
Jul 28 13:16:04.604492 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.604287    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8"
Jul 28 13:16:04.604492 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.604300    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8"
Jul 28 13:16:04.604492 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.604350    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b264fd1a-4ed5-4726-99d3-7fc6299caa49" containerName="switch-graph-c21f8"
Jul 28 13:16:04.608461 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.608442    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:04.611330 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.611309    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-dec07-kube-rbac-proxy-sar-config\""
Jul 28 13:16:04.611454 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.611311    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-dec07-serving-cert\""
Jul 28 13:16:04.618608 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.618580    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"]
Jul 28 13:16:04.741737 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.741699    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00141ead-2a40-4143-8bd6-bfb644c683ff-openshift-service-ca-bundle\") pod \"ensemble-graph-dec07-58484fdcf5-2h8mv\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") " pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:04.741919 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.741751    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00141ead-2a40-4143-8bd6-bfb644c683ff-proxy-tls\") pod \"ensemble-graph-dec07-58484fdcf5-2h8mv\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") " pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:04.842095 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.842057    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00141ead-2a40-4143-8bd6-bfb644c683ff-openshift-service-ca-bundle\") pod \"ensemble-graph-dec07-58484fdcf5-2h8mv\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") " pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:04.842215 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.842111    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00141ead-2a40-4143-8bd6-bfb644c683ff-proxy-tls\") pod \"ensemble-graph-dec07-58484fdcf5-2h8mv\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") " pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:04.842767 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.842742    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00141ead-2a40-4143-8bd6-bfb644c683ff-openshift-service-ca-bundle\") pod \"ensemble-graph-dec07-58484fdcf5-2h8mv\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") " pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:04.844514 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.844494    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00141ead-2a40-4143-8bd6-bfb644c683ff-proxy-tls\") pod \"ensemble-graph-dec07-58484fdcf5-2h8mv\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") " pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:04.919347 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:04.919252    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:05.046802 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:05.046775    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"]
Jul 28 13:16:05.049147 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:16:05.049119    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00141ead_2a40_4143_8bd6_bfb644c683ff.slice/crio-dbb26ac2c3b9a80a9b0210e52237e77bb29c6066ecaf6060661abd54648ed28d WatchSource:0}: Error finding container dbb26ac2c3b9a80a9b0210e52237e77bb29c6066ecaf6060661abd54648ed28d: Status 404 returned error can't find the container with id dbb26ac2c3b9a80a9b0210e52237e77bb29c6066ecaf6060661abd54648ed28d
Jul 28 13:16:05.639222 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:05.639186    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" event={"ID":"00141ead-2a40-4143-8bd6-bfb644c683ff","Type":"ContainerStarted","Data":"fbb72c953f1dd722d38a49a79538a536392b3ff095ac0415cf7b332c036c4449"}
Jul 28 13:16:05.639663 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:05.639238    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" event={"ID":"00141ead-2a40-4143-8bd6-bfb644c683ff","Type":"ContainerStarted","Data":"dbb26ac2c3b9a80a9b0210e52237e77bb29c6066ecaf6060661abd54648ed28d"}
Jul 28 13:16:05.639663 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:05.639308    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:07.847562 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:07.847520    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:10.375134 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.375110    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:16:10.391553 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.391495    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podStartSLOduration=6.391476903 podStartE2EDuration="6.391476903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:16:05.660036734 +0000 UTC m=+1346.305342033" watchObservedRunningTime="2026-07-28 13:16:10.391476903 +0000 UTC m=+1351.036782201"
Jul 28 13:16:10.487576 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.487547    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6485a5-90f6-405b-b73a-17b396cebfc2-openshift-service-ca-bundle\") pod \"7e6485a5-90f6-405b-b73a-17b396cebfc2\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") "
Jul 28 13:16:10.487740 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.487616    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls\") pod \"7e6485a5-90f6-405b-b73a-17b396cebfc2\" (UID: \"7e6485a5-90f6-405b-b73a-17b396cebfc2\") "
Jul 28 13:16:10.487962 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.487940    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e6485a5-90f6-405b-b73a-17b396cebfc2-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "7e6485a5-90f6-405b-b73a-17b396cebfc2" (UID: "7e6485a5-90f6-405b-b73a-17b396cebfc2"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:16:10.489718 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.489689    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7e6485a5-90f6-405b-b73a-17b396cebfc2" (UID: "7e6485a5-90f6-405b-b73a-17b396cebfc2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:16:10.588255 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.588219    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e6485a5-90f6-405b-b73a-17b396cebfc2-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:16:10.588255 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.588249    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6485a5-90f6-405b-b73a-17b396cebfc2-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:16:10.658769 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.658730    2573 generic.go:356] "Generic (PLEG): container finished" podID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerID="1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f" exitCode=0
Jul 28 13:16:10.658957 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.658781    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"
Jul 28 13:16:10.658957 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.658822    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" event={"ID":"7e6485a5-90f6-405b-b73a-17b396cebfc2","Type":"ContainerDied","Data":"1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f"}
Jul 28 13:16:10.658957 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.658886    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj" event={"ID":"7e6485a5-90f6-405b-b73a-17b396cebfc2","Type":"ContainerDied","Data":"e9cb05b2fb3860b923087b26a0ec4ab07c7a1b02c59a2aaa4cff74860e755442"}
Jul 28 13:16:10.658957 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.658918    2573 scope.go:117] "RemoveContainer" containerID="1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f"
Jul 28 13:16:10.669700 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.669671    2573 scope.go:117] "RemoveContainer" containerID="1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f"
Jul 28 13:16:10.670269 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:16:10.670241    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f\": container with ID starting with 1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f not found: ID does not exist" containerID="1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f"
Jul 28 13:16:10.670342 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.670278    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f"} err="failed to get container status \"1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f\": rpc error: code = NotFound desc = could not find container \"1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f\": container with ID starting with 1145742b9d954e60596f3b83046ee84503dafde0d4ac7729d71ca7966b9d333f not found: ID does not exist"
Jul 28 13:16:10.682636 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.682605    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"]
Jul 28 13:16:10.686368 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:10.686341    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fec78-759665677-98xxj"]
Jul 28 13:16:11.651323 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:11.651291    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:11.862677 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:11.862642    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" path="/var/lib/kubelet/pods/7e6485a5-90f6-405b-b73a-17b396cebfc2/volumes"
Jul 28 13:16:14.674508 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:14.674421    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"]
Jul 28 13:16:14.674908 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:14.674604    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07" containerID="cri-o://fbb72c953f1dd722d38a49a79538a536392b3ff095ac0415cf7b332c036c4449" gracePeriod=30
Jul 28 13:16:16.647076 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:16.647031    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:21.647163 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:21.647122    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:26.647542 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:26.647494    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:26.647911 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:26.647609    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:31.647019 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:31.646972    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:36.647004 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:36.646965    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:40.406997 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.406958    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"]
Jul 28 13:16:40.407529 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.407270    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78"
Jul 28 13:16:40.407529 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.407284    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78"
Jul 28 13:16:40.407529 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.407338    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e6485a5-90f6-405b-b73a-17b396cebfc2" containerName="sequence-graph-fec78"
Jul 28 13:16:40.411389 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.411370    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.414232 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.414216    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7ff2e-serving-cert\""
Jul 28 13:16:40.414302 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.414255    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7ff2e-kube-rbac-proxy-sar-config\""
Jul 28 13:16:40.420028 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.419999    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"]
Jul 28 13:16:40.505604 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.505570    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b4bac38-329f-41d2-9cd0-78822db9ddb7-openshift-service-ca-bundle\") pod \"sequence-graph-7ff2e-565db5bccb-wwrvw\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") " pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.505604 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.505611    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b4bac38-329f-41d2-9cd0-78822db9ddb7-proxy-tls\") pod \"sequence-graph-7ff2e-565db5bccb-wwrvw\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") " pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.606279 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.606237    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b4bac38-329f-41d2-9cd0-78822db9ddb7-openshift-service-ca-bundle\") pod \"sequence-graph-7ff2e-565db5bccb-wwrvw\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") " pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.606279 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.606279    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b4bac38-329f-41d2-9cd0-78822db9ddb7-proxy-tls\") pod \"sequence-graph-7ff2e-565db5bccb-wwrvw\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") " pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.606977 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.606949    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b4bac38-329f-41d2-9cd0-78822db9ddb7-openshift-service-ca-bundle\") pod \"sequence-graph-7ff2e-565db5bccb-wwrvw\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") " pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.608762 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.608739    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b4bac38-329f-41d2-9cd0-78822db9ddb7-proxy-tls\") pod \"sequence-graph-7ff2e-565db5bccb-wwrvw\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") " pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.723055 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.722960    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:40.851248 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:40.851214    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"]
Jul 28 13:16:40.854373 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:16:40.854345    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4bac38_329f_41d2_9cd0_78822db9ddb7.slice/crio-4c82ffa4e9727a2015a142d12fadace67ed0483d9593c2f33a3670967b649399 WatchSource:0}: Error finding container 4c82ffa4e9727a2015a142d12fadace67ed0483d9593c2f33a3670967b649399: Status 404 returned error can't find the container with id 4c82ffa4e9727a2015a142d12fadace67ed0483d9593c2f33a3670967b649399
Jul 28 13:16:41.647317 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:41.647281    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:41.772768 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:41.772733    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" event={"ID":"1b4bac38-329f-41d2-9cd0-78822db9ddb7","Type":"ContainerStarted","Data":"b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d"}
Jul 28 13:16:41.772953 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:41.772786    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" event={"ID":"1b4bac38-329f-41d2-9cd0-78822db9ddb7","Type":"ContainerStarted","Data":"4c82ffa4e9727a2015a142d12fadace67ed0483d9593c2f33a3670967b649399"}
Jul 28 13:16:41.772953 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:41.772840    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:41.795433 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:41.791931    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podStartSLOduration=1.79188329 podStartE2EDuration="1.79188329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:16:41.791025095 +0000 UTC m=+1382.436330394" watchObservedRunningTime="2026-07-28 13:16:41.79188329 +0000 UTC m=+1382.437188590"
Jul 28 13:16:44.787757 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:44.787724    2573 generic.go:356] "Generic (PLEG): container finished" podID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerID="fbb72c953f1dd722d38a49a79538a536392b3ff095ac0415cf7b332c036c4449" exitCode=0
Jul 28 13:16:44.788120 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:44.787802    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" event={"ID":"00141ead-2a40-4143-8bd6-bfb644c683ff","Type":"ContainerDied","Data":"fbb72c953f1dd722d38a49a79538a536392b3ff095ac0415cf7b332c036c4449"}
Jul 28 13:16:44.823088 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:44.823064    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:44.938672 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:44.938584    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00141ead-2a40-4143-8bd6-bfb644c683ff-proxy-tls\") pod \"00141ead-2a40-4143-8bd6-bfb644c683ff\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") "
Jul 28 13:16:44.938672 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:44.938624    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00141ead-2a40-4143-8bd6-bfb644c683ff-openshift-service-ca-bundle\") pod \"00141ead-2a40-4143-8bd6-bfb644c683ff\" (UID: \"00141ead-2a40-4143-8bd6-bfb644c683ff\") "
Jul 28 13:16:44.939078 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:44.939048    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00141ead-2a40-4143-8bd6-bfb644c683ff-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "00141ead-2a40-4143-8bd6-bfb644c683ff" (UID: "00141ead-2a40-4143-8bd6-bfb644c683ff"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:16:44.940788 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:44.940762    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00141ead-2a40-4143-8bd6-bfb644c683ff-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "00141ead-2a40-4143-8bd6-bfb644c683ff" (UID: "00141ead-2a40-4143-8bd6-bfb644c683ff"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:16:45.039728 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.039681    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00141ead-2a40-4143-8bd6-bfb644c683ff-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:16:45.039728 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.039718    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00141ead-2a40-4143-8bd6-bfb644c683ff-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:16:45.792662 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.792631    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"
Jul 28 13:16:45.793114 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.792628    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv" event={"ID":"00141ead-2a40-4143-8bd6-bfb644c683ff","Type":"ContainerDied","Data":"dbb26ac2c3b9a80a9b0210e52237e77bb29c6066ecaf6060661abd54648ed28d"}
Jul 28 13:16:45.793114 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.792779    2573 scope.go:117] "RemoveContainer" containerID="fbb72c953f1dd722d38a49a79538a536392b3ff095ac0415cf7b332c036c4449"
Jul 28 13:16:45.814875 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.814842    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"]
Jul 28 13:16:45.818902 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.818877    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-dec07-58484fdcf5-2h8mv"]
Jul 28 13:16:45.862553 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:45.862521    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" path="/var/lib/kubelet/pods/00141ead-2a40-4143-8bd6-bfb644c683ff/volumes"
Jul 28 13:16:47.782560 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:47.782531    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:16:50.490425 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:50.490377    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"]
Jul 28 13:16:50.490814 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:50.490593    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e" containerID="cri-o://b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d" gracePeriod=30
Jul 28 13:16:52.780838 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:52.780799    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:16:57.781291 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:16:57.781250    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:17:02.780941 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:02.780899    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:17:02.781340 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:02.781017    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:17:07.780969 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:07.780929    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:17:12.781209 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:12.781170    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:17:14.925968 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.925903    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"]
Jul 28 13:17:14.926495 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.926225    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07"
Jul 28 13:17:14.926495 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.926242    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07"
Jul 28 13:17:14.926495 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.926348    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="00141ead-2a40-4143-8bd6-bfb644c683ff" containerName="ensemble-graph-dec07"
Jul 28 13:17:14.929529 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.929514    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:14.932557 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.932538    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-7cae9-serving-cert\""
Jul 28 13:17:14.932788 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.932767    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-7cae9-kube-rbac-proxy-sar-config\""
Jul 28 13:17:14.938918 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.938888    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"]
Jul 28 13:17:14.967316 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.967289    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48e5c00-0c3d-4cca-b004-95ec6db85616-openshift-service-ca-bundle\") pod \"ensemble-graph-7cae9-767df688b9-rt9hd\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") " pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:14.967480 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:14.967329    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls\") pod \"ensemble-graph-7cae9-767df688b9-rt9hd\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") " pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:15.068271 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:15.068239    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48e5c00-0c3d-4cca-b004-95ec6db85616-openshift-service-ca-bundle\") pod \"ensemble-graph-7cae9-767df688b9-rt9hd\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") " pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:15.068431 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:15.068277    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls\") pod \"ensemble-graph-7cae9-767df688b9-rt9hd\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") " pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:15.068431 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:17:15.068365    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-7cae9-serving-cert: secret "ensemble-graph-7cae9-serving-cert" not found
Jul 28 13:17:15.068526 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:17:15.068446    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls podName:c48e5c00-0c3d-4cca-b004-95ec6db85616 nodeName:}" failed. No retries permitted until 2026-07-28 13:17:15.56842904 +0000 UTC m=+1416.213734316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls") pod "ensemble-graph-7cae9-767df688b9-rt9hd" (UID: "c48e5c00-0c3d-4cca-b004-95ec6db85616") : secret "ensemble-graph-7cae9-serving-cert" not found
Jul 28 13:17:15.068872 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:15.068853    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48e5c00-0c3d-4cca-b004-95ec6db85616-openshift-service-ca-bundle\") pod \"ensemble-graph-7cae9-767df688b9-rt9hd\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") " pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:15.571866 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:15.571820    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls\") pod \"ensemble-graph-7cae9-767df688b9-rt9hd\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") " pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:15.574171 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:15.574141    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls\") pod \"ensemble-graph-7cae9-767df688b9-rt9hd\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") " pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:15.840361 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:15.840275    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:15.964421 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:15.964371    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"]
Jul 28 13:17:15.967122 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:17:15.967098    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48e5c00_0c3d_4cca_b004_95ec6db85616.slice/crio-a970ab7d9c8c86782a9ba3d02c022b278ccda4511b18093b8db7bd9d6eae1550 WatchSource:0}: Error finding container a970ab7d9c8c86782a9ba3d02c022b278ccda4511b18093b8db7bd9d6eae1550: Status 404 returned error can't find the container with id a970ab7d9c8c86782a9ba3d02c022b278ccda4511b18093b8db7bd9d6eae1550
Jul 28 13:17:16.902821 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:16.902782    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" event={"ID":"c48e5c00-0c3d-4cca-b004-95ec6db85616","Type":"ContainerStarted","Data":"596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5"}
Jul 28 13:17:16.903030 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:16.902876    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" event={"ID":"c48e5c00-0c3d-4cca-b004-95ec6db85616","Type":"ContainerStarted","Data":"a970ab7d9c8c86782a9ba3d02c022b278ccda4511b18093b8db7bd9d6eae1550"}
Jul 28 13:17:16.903030 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:16.902905    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:16.919500 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:16.919450    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podStartSLOduration=2.919435197 podStartE2EDuration="2.919435197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:17:16.918791712 +0000 UTC m=+1417.564097010" watchObservedRunningTime="2026-07-28 13:17:16.919435197 +0000 UTC m=+1417.564740495"
Jul 28 13:17:17.781283 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:17.781238    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:17:20.637868 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.637844    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:17:20.711108 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.711066    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b4bac38-329f-41d2-9cd0-78822db9ddb7-proxy-tls\") pod \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") "
Jul 28 13:17:20.711283 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.711131    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b4bac38-329f-41d2-9cd0-78822db9ddb7-openshift-service-ca-bundle\") pod \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\" (UID: \"1b4bac38-329f-41d2-9cd0-78822db9ddb7\") "
Jul 28 13:17:20.711498 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.711479    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b4bac38-329f-41d2-9cd0-78822db9ddb7-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "1b4bac38-329f-41d2-9cd0-78822db9ddb7" (UID: "1b4bac38-329f-41d2-9cd0-78822db9ddb7"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:17:20.713248 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.713223    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b4bac38-329f-41d2-9cd0-78822db9ddb7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1b4bac38-329f-41d2-9cd0-78822db9ddb7" (UID: "1b4bac38-329f-41d2-9cd0-78822db9ddb7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:17:20.811803 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.811712    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b4bac38-329f-41d2-9cd0-78822db9ddb7-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:17:20.811803 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.811746    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b4bac38-329f-41d2-9cd0-78822db9ddb7-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:17:20.917757 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.917724    2573 generic.go:356] "Generic (PLEG): container finished" podID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerID="b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d" exitCode=0
Jul 28 13:17:20.917941 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.917778    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"
Jul 28 13:17:20.917941 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.917814    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" event={"ID":"1b4bac38-329f-41d2-9cd0-78822db9ddb7","Type":"ContainerDied","Data":"b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d"}
Jul 28 13:17:20.917941 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.917860    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw" event={"ID":"1b4bac38-329f-41d2-9cd0-78822db9ddb7","Type":"ContainerDied","Data":"4c82ffa4e9727a2015a142d12fadace67ed0483d9593c2f33a3670967b649399"}
Jul 28 13:17:20.917941 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.917883    2573 scope.go:117] "RemoveContainer" containerID="b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d"
Jul 28 13:17:20.926102 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.926084    2573 scope.go:117] "RemoveContainer" containerID="b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d"
Jul 28 13:17:20.926363 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:17:20.926346    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d\": container with ID starting with b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d not found: ID does not exist" containerID="b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d"
Jul 28 13:17:20.926436 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.926375    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d"} err="failed to get container status \"b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d\": rpc error: code = NotFound desc = could not find container \"b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d\": container with ID starting with b3632c7bc202a2bf2aaebc42e3995479c21aeb44914b02d61a1b1bc9c880c07d not found: ID does not exist"
Jul 28 13:17:20.939385 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.939351    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"]
Jul 28 13:17:20.942701 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:20.942671    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ff2e-565db5bccb-wwrvw"]
Jul 28 13:17:21.863191 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:21.863154    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" path="/var/lib/kubelet/pods/1b4bac38-329f-41d2-9cd0-78822db9ddb7/volumes"
Jul 28 13:17:22.912828 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:22.912799    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:17:50.790076 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.790042    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"]
Jul 28 13:17:50.790539 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.790300    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e"
Jul 28 13:17:50.790539 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.790310    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e"
Jul 28 13:17:50.790539 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.790354    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4bac38-329f-41d2-9cd0-78822db9ddb7" containerName="sequence-graph-7ff2e"
Jul 28 13:17:50.794675 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.794651    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:50.798301 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.798275    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-dd548-serving-cert\""
Jul 28 13:17:50.798451 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.798275    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-dd548-kube-rbac-proxy-sar-config\""
Jul 28 13:17:50.808479 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.808452    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"]
Jul 28 13:17:50.948540 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.948499    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-openshift-service-ca-bundle\") pod \"sequence-graph-dd548-6dccf7f8d4-xcl74\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") " pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:50.948540 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:50.948547    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls\") pod \"sequence-graph-dd548-6dccf7f8d4-xcl74\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") " pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:51.049674 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:51.049582    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls\") pod \"sequence-graph-dd548-6dccf7f8d4-xcl74\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") " pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:51.049674 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:51.049663    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-openshift-service-ca-bundle\") pod \"sequence-graph-dd548-6dccf7f8d4-xcl74\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") " pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:51.049876 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:17:51.049754    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-dd548-serving-cert: secret "sequence-graph-dd548-serving-cert" not found
Jul 28 13:17:51.049876 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:17:51.049845    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls podName:3fbe842a-36c7-428c-a0f8-f1a18c0b52a3 nodeName:}" failed. No retries permitted until 2026-07-28 13:17:51.549828808 +0000 UTC m=+1452.195134083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls") pod "sequence-graph-dd548-6dccf7f8d4-xcl74" (UID: "3fbe842a-36c7-428c-a0f8-f1a18c0b52a3") : secret "sequence-graph-dd548-serving-cert" not found
Jul 28 13:17:51.050230 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:51.050213    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-openshift-service-ca-bundle\") pod \"sequence-graph-dd548-6dccf7f8d4-xcl74\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") " pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:51.553709 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:51.553663    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls\") pod \"sequence-graph-dd548-6dccf7f8d4-xcl74\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") " pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:51.556125 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:51.556100    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls\") pod \"sequence-graph-dd548-6dccf7f8d4-xcl74\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") " pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:51.704419 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:51.704366    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:51.835378 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:51.835352    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"]
Jul 28 13:17:51.837863 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:17:51.837839    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fbe842a_36c7_428c_a0f8_f1a18c0b52a3.slice/crio-0013b21395b69890d103619de204dd5558f34e9a6578ae738ae0ccf9147c9f2b WatchSource:0}: Error finding container 0013b21395b69890d103619de204dd5558f34e9a6578ae738ae0ccf9147c9f2b: Status 404 returned error can't find the container with id 0013b21395b69890d103619de204dd5558f34e9a6578ae738ae0ccf9147c9f2b
Jul 28 13:17:52.027754 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:52.027717    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" event={"ID":"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3","Type":"ContainerStarted","Data":"ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143"}
Jul 28 13:17:52.027921 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:52.027771    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" event={"ID":"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3","Type":"ContainerStarted","Data":"0013b21395b69890d103619de204dd5558f34e9a6578ae738ae0ccf9147c9f2b"}
Jul 28 13:17:52.027921 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:52.027812    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:17:52.045756 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:52.045703    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podStartSLOduration=2.045688216 podStartE2EDuration="2.045688216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:17:52.044713805 +0000 UTC m=+1452.690019104" watchObservedRunningTime="2026-07-28 13:17:52.045688216 +0000 UTC m=+1452.690993513"
Jul 28 13:17:58.037005 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:17:58.036973    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:25:29.504097 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:29.504060    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"]
Jul 28 13:25:29.506530 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:29.504341    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9" containerID="cri-o://596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5" gracePeriod=30
Jul 28 13:25:32.910392 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:32.910350    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:25:37.910533 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:37.910493    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:25:42.911173 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:42.911128    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:25:42.911575 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:42.911242    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:25:47.911247 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:47.911207    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:25:52.911288 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:52.911248    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:25:57.911004 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:57.910959    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:25:59.650882 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.650858    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:25:59.715013 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.714979    2573 generic.go:356] "Generic (PLEG): container finished" podID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerID="596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5" exitCode=0
Jul 28 13:25:59.715205 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.715026    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"
Jul 28 13:25:59.715205 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.715072    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" event={"ID":"c48e5c00-0c3d-4cca-b004-95ec6db85616","Type":"ContainerDied","Data":"596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5"}
Jul 28 13:25:59.715205 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.715133    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd" event={"ID":"c48e5c00-0c3d-4cca-b004-95ec6db85616","Type":"ContainerDied","Data":"a970ab7d9c8c86782a9ba3d02c022b278ccda4511b18093b8db7bd9d6eae1550"}
Jul 28 13:25:59.715205 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.715155    2573 scope.go:117] "RemoveContainer" containerID="596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5"
Jul 28 13:25:59.722779 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.722761    2573 scope.go:117] "RemoveContainer" containerID="596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5"
Jul 28 13:25:59.723040 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:25:59.723019    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5\": container with ID starting with 596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5 not found: ID does not exist" containerID="596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5"
Jul 28 13:25:59.723105 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.723051    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5"} err="failed to get container status \"596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5\": rpc error: code = NotFound desc = could not find container \"596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5\": container with ID starting with 596a65719665a5e0c426f664bb6657c3d9e8ee3b723fe783969f3d062ea3fce5 not found: ID does not exist"
Jul 28 13:25:59.742896 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.742877    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls\") pod \"c48e5c00-0c3d-4cca-b004-95ec6db85616\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") "
Jul 28 13:25:59.742962 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.742915    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48e5c00-0c3d-4cca-b004-95ec6db85616-openshift-service-ca-bundle\") pod \"c48e5c00-0c3d-4cca-b004-95ec6db85616\" (UID: \"c48e5c00-0c3d-4cca-b004-95ec6db85616\") "
Jul 28 13:25:59.743318 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.743299    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48e5c00-0c3d-4cca-b004-95ec6db85616-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c48e5c00-0c3d-4cca-b004-95ec6db85616" (UID: "c48e5c00-0c3d-4cca-b004-95ec6db85616"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:25:59.744841 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.744822    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c48e5c00-0c3d-4cca-b004-95ec6db85616" (UID: "c48e5c00-0c3d-4cca-b004-95ec6db85616"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:25:59.844061 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.843970    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c48e5c00-0c3d-4cca-b004-95ec6db85616-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:25:59.844061 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:25:59.844004    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48e5c00-0c3d-4cca-b004-95ec6db85616-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:26:00.034969 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:00.034939    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"]
Jul 28 13:26:00.038710 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:00.038685    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7cae9-767df688b9-rt9hd"]
Jul 28 13:26:01.862023 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:01.861985    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" path="/var/lib/kubelet/pods/c48e5c00-0c3d-4cca-b004-95ec6db85616/volumes"
Jul 28 13:26:05.357158 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:05.357126    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"]
Jul 28 13:26:05.357589 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:05.357325    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548" containerID="cri-o://ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143" gracePeriod=30
Jul 28 13:26:08.035632 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:08.035581    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:13.035654 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:13.035614    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:18.035667 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:18.035625    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:18.036058 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:18.035735    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:26:23.036195 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:23.036149    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:28.036187 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:28.036143    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:29.743600 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.743561    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"]
Jul 28 13:26:29.743979 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.743811    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9"
Jul 28 13:26:29.743979 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.743823    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9"
Jul 28 13:26:29.743979 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.743895    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48e5c00-0c3d-4cca-b004-95ec6db85616" containerName="ensemble-graph-7cae9"
Jul 28 13:26:29.746564 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.746547    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:29.749416 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.749381    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-198f4-serving-cert\""
Jul 28 13:26:29.749558 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.749419    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-198f4-kube-rbac-proxy-sar-config\""
Jul 28 13:26:29.757115 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.757089    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"]
Jul 28 13:26:29.875368 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.875333    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-openshift-service-ca-bundle\") pod \"splitter-graph-198f4-5d55f99857-csg76\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") " pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:29.875368 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.875372    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls\") pod \"splitter-graph-198f4-5d55f99857-csg76\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") " pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:29.976505 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.976461    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-openshift-service-ca-bundle\") pod \"splitter-graph-198f4-5d55f99857-csg76\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") " pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:29.976505 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.976510    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls\") pod \"splitter-graph-198f4-5d55f99857-csg76\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") " pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:29.976761 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:26:29.976660    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-198f4-serving-cert: secret "splitter-graph-198f4-serving-cert" not found
Jul 28 13:26:29.976761 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:26:29.976741    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls podName:92072cdc-c46d-4eaf-a849-4ce1bb26ba05 nodeName:}" failed. No retries permitted until 2026-07-28 13:26:30.476717858 +0000 UTC m=+1971.122023135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls") pod "splitter-graph-198f4-5d55f99857-csg76" (UID: "92072cdc-c46d-4eaf-a849-4ce1bb26ba05") : secret "splitter-graph-198f4-serving-cert" not found
Jul 28 13:26:29.977097 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:29.977075    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-openshift-service-ca-bundle\") pod \"splitter-graph-198f4-5d55f99857-csg76\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") " pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:30.480610 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:30.480569    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls\") pod \"splitter-graph-198f4-5d55f99857-csg76\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") " pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:30.482978 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:30.482955    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls\") pod \"splitter-graph-198f4-5d55f99857-csg76\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") " pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:30.656538 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:30.656485    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:30.781629 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:30.781580    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"]
Jul 28 13:26:30.784441 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:26:30.784414    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92072cdc_c46d_4eaf_a849_4ce1bb26ba05.slice/crio-2452c41bdb37a093cd5fbd42e425a3ca3983688116b0951bba37b6c2600b698d WatchSource:0}: Error finding container 2452c41bdb37a093cd5fbd42e425a3ca3983688116b0951bba37b6c2600b698d: Status 404 returned error can't find the container with id 2452c41bdb37a093cd5fbd42e425a3ca3983688116b0951bba37b6c2600b698d
Jul 28 13:26:30.824626 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:30.824596    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" event={"ID":"92072cdc-c46d-4eaf-a849-4ce1bb26ba05","Type":"ContainerStarted","Data":"2452c41bdb37a093cd5fbd42e425a3ca3983688116b0951bba37b6c2600b698d"}
Jul 28 13:26:31.829803 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:31.829718    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" event={"ID":"92072cdc-c46d-4eaf-a849-4ce1bb26ba05","Type":"ContainerStarted","Data":"b9e1a9f9ec41d95164d4ebcfbbc5a75420a4614791924e1a52219042f2cdb6c6"}
Jul 28 13:26:31.830182 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:31.829821    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:31.847911 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:31.847859    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podStartSLOduration=2.847845767 podStartE2EDuration="2.847845767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:26:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:26:31.84632308 +0000 UTC m=+1972.491628369" watchObservedRunningTime="2026-07-28 13:26:31.847845767 +0000 UTC m=+1972.493151064"
Jul 28 13:26:33.035864 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:33.035820    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:35.509741 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.509712    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:26:35.617539 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.617452    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-openshift-service-ca-bundle\") pod \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") "
Jul 28 13:26:35.617539 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.617513    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls\") pod \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\" (UID: \"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3\") "
Jul 28 13:26:35.617822 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.617796    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" (UID: "3fbe842a-36c7-428c-a0f8-f1a18c0b52a3"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:26:35.619535 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.619516    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" (UID: "3fbe842a-36c7-428c-a0f8-f1a18c0b52a3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:26:35.718160 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.718121    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:26:35.718160 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.718156    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:26:35.844233 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.844198    2573 generic.go:356] "Generic (PLEG): container finished" podID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerID="ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143" exitCode=137
Jul 28 13:26:35.844379 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.844245    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"
Jul 28 13:26:35.844379 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.844285    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" event={"ID":"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3","Type":"ContainerDied","Data":"ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143"}
Jul 28 13:26:35.844379 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.844338    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74" event={"ID":"3fbe842a-36c7-428c-a0f8-f1a18c0b52a3","Type":"ContainerDied","Data":"0013b21395b69890d103619de204dd5558f34e9a6578ae738ae0ccf9147c9f2b"}
Jul 28 13:26:35.844379 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.844362    2573 scope.go:117] "RemoveContainer" containerID="ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143"
Jul 28 13:26:35.852447 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.852415    2573 scope.go:117] "RemoveContainer" containerID="ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143"
Jul 28 13:26:35.852730 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:26:35.852705    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143\": container with ID starting with ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143 not found: ID does not exist" containerID="ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143"
Jul 28 13:26:35.852789 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.852742    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143"} err="failed to get container status \"ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143\": rpc error: code = NotFound desc = could not find container \"ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143\": container with ID starting with ba450238c74e5f3015ca145b102166ee6192dd5967d7fcdbdc27cfbefb131143 not found: ID does not exist"
Jul 28 13:26:35.867114 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.867091    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"]
Jul 28 13:26:35.873632 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:35.873585    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-dd548-6dccf7f8d4-xcl74"]
Jul 28 13:26:37.838321 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:37.838291    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:37.863907 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:37.863863    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" path="/var/lib/kubelet/pods/3fbe842a-36c7-428c-a0f8-f1a18c0b52a3/volumes"
Jul 28 13:26:39.829188 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:39.829152    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"]
Jul 28 13:26:39.829576 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:39.829331    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4" containerID="cri-o://b9e1a9f9ec41d95164d4ebcfbbc5a75420a4614791924e1a52219042f2cdb6c6" gracePeriod=30
Jul 28 13:26:42.837858 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:42.837814    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:47.837938 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:47.837893    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:52.838234 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:52.838199    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:52.838636 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:52.838305    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:26:57.837782 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:26:57.837741    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:27:02.837810 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:02.837763    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:27:05.631205 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.631165    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"]
Jul 28 13:27:05.631599 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.631523    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548"
Jul 28 13:27:05.631599 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.631537    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548"
Jul 28 13:27:05.631599 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.631584    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fbe842a-36c7-428c-a0f8-f1a18c0b52a3" containerName="sequence-graph-dd548"
Jul 28 13:27:05.634780 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.634756    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:05.634990 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.634959    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-openshift-service-ca-bundle\") pod \"switch-graph-18d3f-564bd4bccc-zkpnf\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") " pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:05.635065 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.635047    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls\") pod \"switch-graph-18d3f-564bd4bccc-zkpnf\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") " pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:05.637723 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.637700    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-18d3f-serving-cert\""
Jul 28 13:27:05.637831 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.637703    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-18d3f-kube-rbac-proxy-sar-config\""
Jul 28 13:27:05.645459 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.645428    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"]
Jul 28 13:27:05.735679 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.735639    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-openshift-service-ca-bundle\") pod \"switch-graph-18d3f-564bd4bccc-zkpnf\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") " pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:05.735870 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.735693    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls\") pod \"switch-graph-18d3f-564bd4bccc-zkpnf\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") " pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:05.735870 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:27:05.735801    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-18d3f-serving-cert: secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:27:05.735870 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:27:05.735863    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls podName:28b294f2-d9af-43ca-ac08-04e2e13cbeeb nodeName:}" failed. No retries permitted until 2026-07-28 13:27:06.235846508 +0000 UTC m=+2006.881151784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls") pod "switch-graph-18d3f-564bd4bccc-zkpnf" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb") : secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:27:05.736265 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:05.736246    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-openshift-service-ca-bundle\") pod \"switch-graph-18d3f-564bd4bccc-zkpnf\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") " pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:06.239087 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.239048    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls\") pod \"switch-graph-18d3f-564bd4bccc-zkpnf\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") " pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:06.241500 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.241465    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls\") pod \"switch-graph-18d3f-564bd4bccc-zkpnf\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") " pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:06.245292 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.245272    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:06.380090 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.380036    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"]
Jul 28 13:27:06.382272 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:27:06.382244    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b294f2_d9af_43ca_ac08_04e2e13cbeeb.slice/crio-f2cac4b57318865fd598b2decc801987de70de720b23c3b11ad8586d93d78395 WatchSource:0}: Error finding container f2cac4b57318865fd598b2decc801987de70de720b23c3b11ad8586d93d78395: Status 404 returned error can't find the container with id f2cac4b57318865fd598b2decc801987de70de720b23c3b11ad8586d93d78395
Jul 28 13:27:06.956421 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.956351    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" event={"ID":"28b294f2-d9af-43ca-ac08-04e2e13cbeeb","Type":"ContainerStarted","Data":"9e2ff7dc7a12f975adb7e7be240917a597bccb6831b0dace0757b6dd94cb1d87"}
Jul 28 13:27:06.956809 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.956440    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" event={"ID":"28b294f2-d9af-43ca-ac08-04e2e13cbeeb","Type":"ContainerStarted","Data":"f2cac4b57318865fd598b2decc801987de70de720b23c3b11ad8586d93d78395"}
Jul 28 13:27:06.956809 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.956470    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:06.974465 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:06.974392    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podStartSLOduration=1.974373977 podStartE2EDuration="1.974373977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:27:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:27:06.972947149 +0000 UTC m=+2007.618252449" watchObservedRunningTime="2026-07-28 13:27:06.974373977 +0000 UTC m=+2007.619679276"
Jul 28 13:27:07.838107 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:07.838060    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:27:09.968259 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:09.968231    2573 generic.go:356] "Generic (PLEG): container finished" podID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerID="b9e1a9f9ec41d95164d4ebcfbbc5a75420a4614791924e1a52219042f2cdb6c6" exitCode=0
Jul 28 13:27:09.968606 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:09.968303    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" event={"ID":"92072cdc-c46d-4eaf-a849-4ce1bb26ba05","Type":"ContainerDied","Data":"b9e1a9f9ec41d95164d4ebcfbbc5a75420a4614791924e1a52219042f2cdb6c6"}
Jul 28 13:27:09.968606 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:09.968359    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76" event={"ID":"92072cdc-c46d-4eaf-a849-4ce1bb26ba05","Type":"ContainerDied","Data":"2452c41bdb37a093cd5fbd42e425a3ca3983688116b0951bba37b6c2600b698d"}
Jul 28 13:27:09.968606 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:09.968376    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2452c41bdb37a093cd5fbd42e425a3ca3983688116b0951bba37b6c2600b698d"
Jul 28 13:27:09.979085 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:09.979060    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:27:10.164754 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.164655    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-openshift-service-ca-bundle\") pod \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") "
Jul 28 13:27:10.164754 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.164695    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls\") pod \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\" (UID: \"92072cdc-c46d-4eaf-a849-4ce1bb26ba05\") "
Jul 28 13:27:10.165079 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.165055    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "92072cdc-c46d-4eaf-a849-4ce1bb26ba05" (UID: "92072cdc-c46d-4eaf-a849-4ce1bb26ba05"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:27:10.166801 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.166778    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "92072cdc-c46d-4eaf-a849-4ce1bb26ba05" (UID: "92072cdc-c46d-4eaf-a849-4ce1bb26ba05"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:27:10.265538 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.265502    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:27:10.265538 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.265533    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92072cdc-c46d-4eaf-a849-4ce1bb26ba05-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:27:10.973435 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.972833    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"
Jul 28 13:27:10.997643 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:10.997607    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"]
Jul 28 13:27:11.001207 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:11.001179    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-198f4-5d55f99857-csg76"]
Jul 28 13:27:11.862495 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:11.862459    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" path="/var/lib/kubelet/pods/92072cdc-c46d-4eaf-a849-4ce1bb26ba05/volumes"
Jul 28 13:27:12.965724 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:12.965698    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:27:40.073614 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.073579    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"]
Jul 28 13:27:40.074143 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.073949    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4"
Jul 28 13:27:40.074143 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.073970    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4"
Jul 28 13:27:40.074143 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.074038    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="92072cdc-c46d-4eaf-a849-4ce1bb26ba05" containerName="splitter-graph-198f4"
Jul 28 13:27:40.077341 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.077322    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.080192 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.080167    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-b971b-serving-cert\""
Jul 28 13:27:40.080604 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.080590    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-b971b-kube-rbac-proxy-sar-config\""
Jul 28 13:27:40.085873 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.085847    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"]
Jul 28 13:27:40.174927 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.174864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls\") pod \"splitter-graph-b971b-bb5445b6c-lfxks\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") " pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.175116 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.175004    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1890d911-17dd-4328-b062-344a18050a90-openshift-service-ca-bundle\") pod \"splitter-graph-b971b-bb5445b6c-lfxks\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") " pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.275591 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.275549    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls\") pod \"splitter-graph-b971b-bb5445b6c-lfxks\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") " pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.275799 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.275625    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1890d911-17dd-4328-b062-344a18050a90-openshift-service-ca-bundle\") pod \"splitter-graph-b971b-bb5445b6c-lfxks\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") " pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.275799 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:27:40.275708    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-b971b-serving-cert: secret "splitter-graph-b971b-serving-cert" not found
Jul 28 13:27:40.275799 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:27:40.275792    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls podName:1890d911-17dd-4328-b062-344a18050a90 nodeName:}" failed. No retries permitted until 2026-07-28 13:27:40.775771117 +0000 UTC m=+2041.421076401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls") pod "splitter-graph-b971b-bb5445b6c-lfxks" (UID: "1890d911-17dd-4328-b062-344a18050a90") : secret "splitter-graph-b971b-serving-cert" not found
Jul 28 13:27:40.276324 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.276303    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1890d911-17dd-4328-b062-344a18050a90-openshift-service-ca-bundle\") pod \"splitter-graph-b971b-bb5445b6c-lfxks\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") " pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.779893 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.779848    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls\") pod \"splitter-graph-b971b-bb5445b6c-lfxks\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") " pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.782320 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.782297    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls\") pod \"splitter-graph-b971b-bb5445b6c-lfxks\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") " pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:40.988671 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:40.988631    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:41.120048 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:41.120022    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"]
Jul 28 13:27:41.122695 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:27:41.122671    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1890d911_17dd_4328_b062_344a18050a90.slice/crio-63775876dda4b7c3d8e3c3f3c3155bfcd68e1b4c84c17dac4d87a67827b133d4 WatchSource:0}: Error finding container 63775876dda4b7c3d8e3c3f3c3155bfcd68e1b4c84c17dac4d87a67827b133d4: Status 404 returned error can't find the container with id 63775876dda4b7c3d8e3c3f3c3155bfcd68e1b4c84c17dac4d87a67827b133d4
Jul 28 13:27:42.082859 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:42.082826    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" event={"ID":"1890d911-17dd-4328-b062-344a18050a90","Type":"ContainerStarted","Data":"7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f"}
Jul 28 13:27:42.083045 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:42.082879    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" event={"ID":"1890d911-17dd-4328-b062-344a18050a90","Type":"ContainerStarted","Data":"63775876dda4b7c3d8e3c3f3c3155bfcd68e1b4c84c17dac4d87a67827b133d4"}
Jul 28 13:27:42.083045 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:42.082950    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:27:42.101536 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:42.101495    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podStartSLOduration=2.101482102 podStartE2EDuration="2.101482102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:27:42.099111066 +0000 UTC m=+2042.744416364" watchObservedRunningTime="2026-07-28 13:27:42.101482102 +0000 UTC m=+2042.746787452"
Jul 28 13:27:48.091569 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:27:48.091535    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:32:39.932932 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:32:39.932867    2573 scope.go:117] "RemoveContainer" containerID="b9e1a9f9ec41d95164d4ebcfbbc5a75420a4614791924e1a52219042f2cdb6c6"
Jul 28 13:35:54.747724 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:35:54.747686    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"]
Jul 28 13:35:54.750221 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:35:54.747944    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b" containerID="cri-o://7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f" gracePeriod=30
Jul 28 13:35:58.089969 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:35:58.089919    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:36:03.090143 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:03.090104    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:36:08.090492 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:08.090453    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:36:08.090986 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:08.090580    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:36:13.091004 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:13.090963    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:36:18.090171 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:18.090132    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:36:23.090839 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:23.090802    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:36:24.879822 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.879795    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:36:24.888561 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.888536    2573 generic.go:356] "Generic (PLEG): container finished" podID="1890d911-17dd-4328-b062-344a18050a90" containerID="7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f" exitCode=0
Jul 28 13:36:24.888661 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.888571    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"
Jul 28 13:36:24.888661 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.888588    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" event={"ID":"1890d911-17dd-4328-b062-344a18050a90","Type":"ContainerDied","Data":"7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f"}
Jul 28 13:36:24.888661 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.888648    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks" event={"ID":"1890d911-17dd-4328-b062-344a18050a90","Type":"ContainerDied","Data":"63775876dda4b7c3d8e3c3f3c3155bfcd68e1b4c84c17dac4d87a67827b133d4"}
Jul 28 13:36:24.888761 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.888671    2573 scope.go:117] "RemoveContainer" containerID="7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f"
Jul 28 13:36:24.896146 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.896130    2573 scope.go:117] "RemoveContainer" containerID="7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f"
Jul 28 13:36:24.896377 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:36:24.896356    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f\": container with ID starting with 7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f not found: ID does not exist" containerID="7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f"
Jul 28 13:36:24.896462 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:24.896389    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f"} err="failed to get container status \"7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f\": rpc error: code = NotFound desc = could not find container \"7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f\": container with ID starting with 7d22c98dd2abb5cd139f6932b499f261f64c8433fa974b86fb6b219e3d7d575f not found: ID does not exist"
Jul 28 13:36:25.044957 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.044868    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls\") pod \"1890d911-17dd-4328-b062-344a18050a90\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") "
Jul 28 13:36:25.044957 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.044921    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1890d911-17dd-4328-b062-344a18050a90-openshift-service-ca-bundle\") pod \"1890d911-17dd-4328-b062-344a18050a90\" (UID: \"1890d911-17dd-4328-b062-344a18050a90\") "
Jul 28 13:36:25.045290 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.045264    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1890d911-17dd-4328-b062-344a18050a90-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "1890d911-17dd-4328-b062-344a18050a90" (UID: "1890d911-17dd-4328-b062-344a18050a90"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:36:25.046896 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.046869    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1890d911-17dd-4328-b062-344a18050a90" (UID: "1890d911-17dd-4328-b062-344a18050a90"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:36:25.146115 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.146085    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1890d911-17dd-4328-b062-344a18050a90-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:36:25.146115 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.146112    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1890d911-17dd-4328-b062-344a18050a90-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:36:25.211436 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.211387    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"]
Jul 28 13:36:25.215382 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.215356    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b971b-bb5445b6c-lfxks"]
Jul 28 13:36:25.863049 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:36:25.863013    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1890d911-17dd-4328-b062-344a18050a90" path="/var/lib/kubelet/pods/1890d911-17dd-4328-b062-344a18050a90/volumes"
Jul 28 13:43:24.960147 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:24.960111    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-18d3f-serving-cert: secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:24.960872 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:24.960196    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls podName:28b294f2-d9af-43ca-ac08-04e2e13cbeeb nodeName:}" failed. No retries permitted until 2026-07-28 13:43:25.460177128 +0000 UTC m=+2986.105482406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls") pod "switch-graph-18d3f-564bd4bccc-zkpnf" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb") : secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:24.965223 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:24.965194    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"]
Jul 28 13:43:25.316842 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:25.316800    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f" containerID="cri-o://9e2ff7dc7a12f975adb7e7be240917a597bccb6831b0dace0757b6dd94cb1d87" gracePeriod=30
Jul 28 13:43:25.464834 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:25.464798    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-18d3f-serving-cert: secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:25.465001 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:25.464863    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls podName:28b294f2-d9af-43ca-ac08-04e2e13cbeeb nodeName:}" failed. No retries permitted until 2026-07-28 13:43:26.464849342 +0000 UTC m=+2987.110154618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls") pod "switch-graph-18d3f-564bd4bccc-zkpnf" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb") : secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:26.473699 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:26.473644    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-18d3f-serving-cert: secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:26.474106 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:26.473734    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls podName:28b294f2-d9af-43ca-ac08-04e2e13cbeeb nodeName:}" failed. No retries permitted until 2026-07-28 13:43:28.473718743 +0000 UTC m=+2989.119024019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls") pod "switch-graph-18d3f-564bd4bccc-zkpnf" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb") : secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:27.964673 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:27.964627    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:43:28.489003 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:28.488975    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-18d3f-serving-cert: secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:28.489172 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:28.489048    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls podName:28b294f2-d9af-43ca-ac08-04e2e13cbeeb nodeName:}" failed. No retries permitted until 2026-07-28 13:43:32.489031779 +0000 UTC m=+2993.134337071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls") pod "switch-graph-18d3f-564bd4bccc-zkpnf" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb") : secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:32.522261 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:32.522227    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-18d3f-serving-cert: secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:32.522678 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:32.522293    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls podName:28b294f2-d9af-43ca-ac08-04e2e13cbeeb nodeName:}" failed. No retries permitted until 2026-07-28 13:43:40.522280669 +0000 UTC m=+3001.167585945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls") pod "switch-graph-18d3f-564bd4bccc-zkpnf" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb") : secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:32.964591 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:32.964500    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:43:37.964017 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:37.963973    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:43:37.964416 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:37.964080    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:43:40.586063 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:40.586023    2573 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-18d3f-serving-cert: secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:40.586491 ip-10-0-132-6 kubenswrapper[2573]: E0728 13:43:40.586103    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls podName:28b294f2-d9af-43ca-ac08-04e2e13cbeeb nodeName:}" failed. No retries permitted until 2026-07-28 13:43:56.58608737 +0000 UTC m=+3017.231392646 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls") pod "switch-graph-18d3f-564bd4bccc-zkpnf" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb") : secret "switch-graph-18d3f-serving-cert" not found
Jul 28 13:43:41.337523 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:41.337491    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:42.160931 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:42.160899    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:42.963860 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:42.963820    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:43:42.994351 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:42.994318    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:43.786141 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:43.786110    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:44.568250 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:44.568220    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:45.364332 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:45.364299    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:46.201390 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:46.201349    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:46.991412 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:46.991374    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:47.808241 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:47.808207    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:47.964550 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:47.964504    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:43:48.590894 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:48.590860    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:49.380967 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:49.380933    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:50.154358 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:50.154328    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:50.960416 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:50.960372    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:51.795844 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:51.795813    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-18d3f-564bd4bccc-zkpnf_28b294f2-d9af-43ca-ac08-04e2e13cbeeb/switch-graph-18d3f/0.log"
Jul 28 13:43:52.964572 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:52.964525    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:43:55.428247 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.428214    2573 generic.go:356] "Generic (PLEG): container finished" podID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerID="9e2ff7dc7a12f975adb7e7be240917a597bccb6831b0dace0757b6dd94cb1d87" exitCode=0
Jul 28 13:43:55.428652 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.428284    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" event={"ID":"28b294f2-d9af-43ca-ac08-04e2e13cbeeb","Type":"ContainerDied","Data":"9e2ff7dc7a12f975adb7e7be240917a597bccb6831b0dace0757b6dd94cb1d87"}
Jul 28 13:43:55.463950 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.463922    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:43:55.504465 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.504433    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-openshift-service-ca-bundle\") pod \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") "
Jul 28 13:43:55.504629 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.504513    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls\") pod \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\" (UID: \"28b294f2-d9af-43ca-ac08-04e2e13cbeeb\") "
Jul 28 13:43:55.504827 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.504804    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "28b294f2-d9af-43ca-ac08-04e2e13cbeeb" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:43:55.506551 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.506517    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "28b294f2-d9af-43ca-ac08-04e2e13cbeeb" (UID: "28b294f2-d9af-43ca-ac08-04e2e13cbeeb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:43:55.605587 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.605481    2573 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-proxy-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:43:55.605587 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:55.605527    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b294f2-d9af-43ca-ac08-04e2e13cbeeb-openshift-service-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\""
Jul 28 13:43:56.353195 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.353157    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-95qqx/must-gather-xp5tn"]
Jul 28 13:43:56.353476 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.353409    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b"
Jul 28 13:43:56.353476 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.353420    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b"
Jul 28 13:43:56.353476 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.353432    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f"
Jul 28 13:43:56.353476 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.353438    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f"
Jul 28 13:43:56.353625 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.353483    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1890d911-17dd-4328-b062-344a18050a90" containerName="splitter-graph-b971b"
Jul 28 13:43:56.353625 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.353492    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" containerName="switch-graph-18d3f"
Jul 28 13:43:56.356445 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.356428    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.359937 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.359905    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-95qqx\"/\"openshift-service-ca.crt\""
Jul 28 13:43:56.359937 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.359927    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-95qqx\"/\"default-dockercfg-7wr6f\""
Jul 28 13:43:56.361244 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.361227    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-95qqx\"/\"kube-root-ca.crt\""
Jul 28 13:43:56.367664 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.367641    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95qqx/must-gather-xp5tn"]
Jul 28 13:43:56.411280 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.411250    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6s9z\" (UniqueName: \"kubernetes.io/projected/1c8458de-6f5a-461d-861e-386d8028cc83-kube-api-access-s6s9z\") pod \"must-gather-xp5tn\" (UID: \"1c8458de-6f5a-461d-861e-386d8028cc83\") " pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.411493 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.411341    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1c8458de-6f5a-461d-861e-386d8028cc83-must-gather-output\") pod \"must-gather-xp5tn\" (UID: \"1c8458de-6f5a-461d-861e-386d8028cc83\") " pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.433080 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.433047    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"
Jul 28 13:43:56.433080 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.433071    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf" event={"ID":"28b294f2-d9af-43ca-ac08-04e2e13cbeeb","Type":"ContainerDied","Data":"f2cac4b57318865fd598b2decc801987de70de720b23c3b11ad8586d93d78395"}
Jul 28 13:43:56.433534 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.433136    2573 scope.go:117] "RemoveContainer" containerID="9e2ff7dc7a12f975adb7e7be240917a597bccb6831b0dace0757b6dd94cb1d87"
Jul 28 13:43:56.450638 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.450605    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"]
Jul 28 13:43:56.454683 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.454663    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-18d3f-564bd4bccc-zkpnf"]
Jul 28 13:43:56.512366 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.512336    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6s9z\" (UniqueName: \"kubernetes.io/projected/1c8458de-6f5a-461d-861e-386d8028cc83-kube-api-access-s6s9z\") pod \"must-gather-xp5tn\" (UID: \"1c8458de-6f5a-461d-861e-386d8028cc83\") " pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.512554 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.512419    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1c8458de-6f5a-461d-861e-386d8028cc83-must-gather-output\") pod \"must-gather-xp5tn\" (UID: \"1c8458de-6f5a-461d-861e-386d8028cc83\") " pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.512699 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.512684    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1c8458de-6f5a-461d-861e-386d8028cc83-must-gather-output\") pod \"must-gather-xp5tn\" (UID: \"1c8458de-6f5a-461d-861e-386d8028cc83\") " pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.522877 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.522845    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6s9z\" (UniqueName: \"kubernetes.io/projected/1c8458de-6f5a-461d-861e-386d8028cc83-kube-api-access-s6s9z\") pod \"must-gather-xp5tn\" (UID: \"1c8458de-6f5a-461d-861e-386d8028cc83\") " pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.665593 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.665495    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95qqx/must-gather-xp5tn"
Jul 28 13:43:56.797703 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.797560    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95qqx/must-gather-xp5tn"]
Jul 28 13:43:56.800327 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:43:56.800287    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c8458de_6f5a_461d_861e_386d8028cc83.slice/crio-e1e53c5ca4659ca9106f4a40a948e84bf4fcef8516fc06e0f53744481e2190f1 WatchSource:0}: Error finding container e1e53c5ca4659ca9106f4a40a948e84bf4fcef8516fc06e0f53744481e2190f1: Status 404 returned error can't find the container with id e1e53c5ca4659ca9106f4a40a948e84bf4fcef8516fc06e0f53744481e2190f1
Jul 28 13:43:56.802137 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:56.802121    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:43:57.437905 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:57.437862    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95qqx/must-gather-xp5tn" event={"ID":"1c8458de-6f5a-461d-861e-386d8028cc83","Type":"ContainerStarted","Data":"e1e53c5ca4659ca9106f4a40a948e84bf4fcef8516fc06e0f53744481e2190f1"}
Jul 28 13:43:57.862608 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:57.862576    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28b294f2-d9af-43ca-ac08-04e2e13cbeeb" path="/var/lib/kubelet/pods/28b294f2-d9af-43ca-ac08-04e2e13cbeeb/volumes"
Jul 28 13:43:58.447196 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:58.447158    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95qqx/must-gather-xp5tn" event={"ID":"1c8458de-6f5a-461d-861e-386d8028cc83","Type":"ContainerStarted","Data":"44acbf6212a0769e99190e463ea8ab0d903e60e6ffd2a4b24c1444290f6bb064"}
Jul 28 13:43:58.447678 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:58.447234    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95qqx/must-gather-xp5tn" event={"ID":"1c8458de-6f5a-461d-861e-386d8028cc83","Type":"ContainerStarted","Data":"2aa071f0bf59347b83436181ed40096005ab75f52210f95641bdf73b348ef393"}
Jul 28 13:43:58.979220 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:58.979177    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6vk6b_282de852-6184-4004-a1fd-689bd1949d36/global-pull-secret-syncer/0.log"
Jul 28 13:43:59.161004 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:59.160972    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s5s82_323970a5-f292-4c73-82c5-74b4cf765494/konnectivity-agent/0.log"
Jul 28 13:43:59.189953 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:43:59.189921    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-6.ec2.internal_536c5ff9fffd6975f95e3ba4d3456e6e/haproxy/0.log"
Jul 28 13:44:03.996657 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:03.996611    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zdbdl_06e3413b-1a24-4f95-b0cc-0d02c0bde21a/node-exporter/0.log"
Jul 28 13:44:04.026176 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:04.026142    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zdbdl_06e3413b-1a24-4f95-b0cc-0d02c0bde21a/kube-rbac-proxy/0.log"
Jul 28 13:44:04.062042 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:04.061983    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zdbdl_06e3413b-1a24-4f95-b0cc-0d02c0bde21a/init-textfile/0.log"
Jul 28 13:44:06.816156 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:06.816079    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75b4949448-4d55d_d490961f-108d-45b4-a2bb-175ef60f65d7/console/0.log"
Jul 28 13:44:07.263661 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.263595    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-95qqx/must-gather-xp5tn" podStartSLOduration=10.462720564 podStartE2EDuration="11.263571712s" totalImagesPullingTime="800.851148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:43:58.467422757 +0000 UTC m=+3019.112728056" watchObservedRunningTime="2026-07-28 13:44:07.263571712 +0000 UTC m=+3027.908877008"
Jul 28 13:44:07.265219 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.265181    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"]
Jul 28 13:44:07.269152 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.269129    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.277990 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.277959    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"]
Jul 28 13:44:07.406732 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.406672    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-lib-modules\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.406898 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.406745    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p6xp\" (UniqueName: \"kubernetes.io/projected/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-kube-api-access-2p6xp\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.406898 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.406838    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-proc\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.406898 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.406869    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-podres\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.407018 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.406909    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-sys\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.507938 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.507898    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-proc\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508208 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508182    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-proc\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508305 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508189    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-podres\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508305 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508289    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-sys\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508443 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508373    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-lib-modules\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508443 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508424    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p6xp\" (UniqueName: \"kubernetes.io/projected/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-kube-api-access-2p6xp\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508443 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508431    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-sys\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508606 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508555    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-lib-modules\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.508808 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.508776    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-podres\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.518113 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.518027    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p6xp\" (UniqueName: \"kubernetes.io/projected/8f3565d5-69e4-444c-b6b2-ee08b30d0df0-kube-api-access-2p6xp\") pod \"perf-node-gather-daemonset-cv65z\" (UID: \"8f3565d5-69e4-444c-b6b2-ee08b30d0df0\") " pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.582772 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.582741    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:07.753756 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:07.753576    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"]
Jul 28 13:44:07.761089 ip-10-0-132-6 kubenswrapper[2573]: W0728 13:44:07.761053    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8f3565d5_69e4_444c_b6b2_ee08b30d0df0.slice/crio-d41bc2cc336a23c81f10c1329d3805cb0607a81e907143cfbe60a7cca2897c5a WatchSource:0}: Error finding container d41bc2cc336a23c81f10c1329d3805cb0607a81e907143cfbe60a7cca2897c5a: Status 404 returned error can't find the container with id d41bc2cc336a23c81f10c1329d3805cb0607a81e907143cfbe60a7cca2897c5a
Jul 28 13:44:08.336075 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.336044    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h9zqs_f82883b2-4aec-4ac2-a169-96e59b1cdae8/dns/0.log"
Jul 28 13:44:08.356439 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.356392    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h9zqs_f82883b2-4aec-4ac2-a169-96e59b1cdae8/kube-rbac-proxy/0.log"
Jul 28 13:44:08.381059 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.381030    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sr76w_71951990-b81f-4883-94f4-cd91e45251d2/dns-node-resolver/0.log"
Jul 28 13:44:08.504173 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.504131    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z" event={"ID":"8f3565d5-69e4-444c-b6b2-ee08b30d0df0","Type":"ContainerStarted","Data":"c70fa02676800a5bdcb1fbb4f2d581527a83a11b94f059c02d75025b9ba1fc32"}
Jul 28 13:44:08.504372 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.504205    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z" event={"ID":"8f3565d5-69e4-444c-b6b2-ee08b30d0df0","Type":"ContainerStarted","Data":"d41bc2cc336a23c81f10c1329d3805cb0607a81e907143cfbe60a7cca2897c5a"}
Jul 28 13:44:08.505026 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.505000    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:08.526623 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.526560    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z" podStartSLOduration=1.526538154 podStartE2EDuration="1.526538154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:44:08.526035269 +0000 UTC m=+3029.171340568" watchObservedRunningTime="2026-07-28 13:44:08.526538154 +0000 UTC m=+3029.171843453"
Jul 28 13:44:08.882713 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.882661    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-79979f7559-dh4c9_36876a04-8485-46b2-bf64-b916527acafe/registry/0.log"
Jul 28 13:44:08.906881 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:08.906848    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bzkx5_b89a2f2a-171a-40d2-b23f-1523973a6329/node-ca/0.log"
Jul 28 13:44:09.992516 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:09.992478    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hhvlv_32827ef9-a4c6-4225-a864-097970885c5e/serve-healthcheck-canary/0.log"
Jul 28 13:44:10.456213 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:10.456120    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qsbvw_72bdd2c4-2110-465a-a54b-49c04ae93fa3/kube-rbac-proxy/0.log"
Jul 28 13:44:10.477231 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:10.477204    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qsbvw_72bdd2c4-2110-465a-a54b-49c04ae93fa3/exporter/0.log"
Jul 28 13:44:10.498895 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:10.498863    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qsbvw_72bdd2c4-2110-465a-a54b-49c04ae93fa3/extractor/0.log"
Jul 28 13:44:12.821807 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:12.821773    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-84fc959947-52q9v_34d6af1c-2eca-4dad-909d-99b06669a84b/manager/0.log"
Jul 28 13:44:15.521119 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:15.521094    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-95qqx/perf-node-gather-daemonset-cv65z"
Jul 28 13:44:18.619820 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.619791    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdhtq_bea7b828-fe51-446f-a271-8900e62c55a3/kube-multus-additional-cni-plugins/0.log"
Jul 28 13:44:18.641598 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.641573    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdhtq_bea7b828-fe51-446f-a271-8900e62c55a3/egress-router-binary-copy/0.log"
Jul 28 13:44:18.663031 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.662997    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdhtq_bea7b828-fe51-446f-a271-8900e62c55a3/cni-plugins/0.log"
Jul 28 13:44:18.685251 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.685223    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdhtq_bea7b828-fe51-446f-a271-8900e62c55a3/bond-cni-plugin/0.log"
Jul 28 13:44:18.707089 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.707065    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdhtq_bea7b828-fe51-446f-a271-8900e62c55a3/routeoverride-cni/0.log"
Jul 28 13:44:18.730130 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.730100    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdhtq_bea7b828-fe51-446f-a271-8900e62c55a3/whereabouts-cni-bincopy/0.log"
Jul 28 13:44:18.752626 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.752546    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdhtq_bea7b828-fe51-446f-a271-8900e62c55a3/whereabouts-cni/0.log"
Jul 28 13:44:18.983327 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:18.983291    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h27hs_2b1be2bf-c833-4639-9f5e-f7621efdcb9d/kube-multus/0.log"
Jul 28 13:44:19.129803 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:19.129710    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r8nsf_49c7f34f-5a6c-4986-93e4-16ff1577d508/network-metrics-daemon/0.log"
Jul 28 13:44:19.150340 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:19.150307    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r8nsf_49c7f34f-5a6c-4986-93e4-16ff1577d508/kube-rbac-proxy/0.log"
Jul 28 13:44:20.485348 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.485314    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/ovn-controller/0.log"
Jul 28 13:44:20.502564 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.502532    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/ovn-acl-logging/0.log"
Jul 28 13:44:20.534096 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.534064    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/kube-rbac-proxy-node/0.log"
Jul 28 13:44:20.555178 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.555150    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/kube-rbac-proxy-ovn-metrics/0.log"
Jul 28 13:44:20.575226 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.575178    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/northd/0.log"
Jul 28 13:44:20.596631 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.596594    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/nbdb/0.log"
Jul 28 13:44:20.619722 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.619688    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/sbdb/0.log"
Jul 28 13:44:20.684570 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:20.684530    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vr6js_8d74a432-2d04-4ccb-bd11-2ff510789087/ovnkube-controller/0.log"
Jul 28 13:44:21.669892 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:21.669862    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-9qcmh_3499e17e-83e5-40a6-9e3e-b1684607cfe1/network-check-target-container/0.log"
Jul 28 13:44:22.649747 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:22.649705    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-vcl8j_d3d01e36-e904-4ef1-bd3e-e2ac5b99660a/iptables-alerter/0.log"
Jul 28 13:44:23.268944 ip-10-0-132-6 kubenswrapper[2573]: I0728 13:44:23.268899    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-qv6xh_0e54284b-26d3-42a4-8b8a-a6c1c6c202d8/tuned/0.log"