Jul 07 14:56:22.531058 ip-10-0-134-178 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.979470    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984780    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984812    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984816    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984819    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984823    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984826    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984829    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:23.057008 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984832    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984835    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984837    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984840    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984843    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984846    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984848    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984851    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984853    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984856    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984859    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984862    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984865    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984868    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984870    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984873    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984875    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984878    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984884    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984888    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:23.062722 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984890    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984893    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984896    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984899    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984901    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984904    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984908    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984912    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984915    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984918    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984922    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984924    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984927    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984930    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984932    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984935    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984937    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984940    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984943    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984945    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:23.063436 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984948    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:23.063727 ip-10-0-134-178 systemd[1]: Started Kubernetes Kubelet.
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984950    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984953    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984958    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984962    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984966    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984969    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984972    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984976    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984979    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984983    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984985    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984988    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984991    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984994    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984997    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.984999    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985002    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985005    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:23.064729 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985007    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985010    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985013    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985015    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985018    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985020    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985023    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985026    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985028    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985031    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985034    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985036    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985039    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985042    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985045    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985047    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985050    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985053    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985055    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:23.066251 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985058    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985060    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985064    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985068    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985070    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985073    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985076    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985079    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985082    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985084    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985087    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985089    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985092    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985095    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985814    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985821    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985824    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985827    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985832    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985834    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:23.067432 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985837    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985840    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985843    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985846    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985848    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985852    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985855    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985858    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985861    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985863    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985866    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985869    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985872    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985874    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985877    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985880    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985882    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985885    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985888    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985890    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:23.068282 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985893    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985896    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985898    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985901    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985904    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985907    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985909    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985912    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985914    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985917    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985920    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985923    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985925    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985928    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985931    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985936    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985939    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985943    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985946    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:23.069113 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985949    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985951    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985954    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985956    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985959    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985963    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985965    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985968    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985971    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985974    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985977    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985979    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985984    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985987    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985990    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985992    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985995    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.985997    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986000    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986002    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:23.069691 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986006    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986009    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986011    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986014    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986016    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986019    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986021    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986026    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986029    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986032    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986035    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986038    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986040    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986043    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986047    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986050    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986054    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986057    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986060    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:23.070545 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986063    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986065    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986068    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986071    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986074    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986079    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986082    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986084    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986087    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986089    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986092    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986094    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986097    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986100    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986102    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986542    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986548    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986551    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986554    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986557    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:23.071169 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986560    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986564    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986566    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986569    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986572    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986575    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986578    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986580    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986583    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986585    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986590    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986593    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986596    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986599    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986601    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986604    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986606    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986610    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986614    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:23.071809 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986616    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986619    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986621    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986624    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986626    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986629    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986632    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986634    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986637    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986639    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986641    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986644    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986647    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986649    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986652    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986654    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986657    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986659    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986661    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986664    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:23.135385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986669    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986673    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986676    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986681    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986684    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986694    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986697    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986713    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986717    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986720    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986724    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986727    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986730    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986733    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986736    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986738    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986741    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986744    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986746    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:23.136138 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986749    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986754    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986757    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986760    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986763    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986766    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986768    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986771    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986774    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986777    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986779    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986781    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986784    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986788    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986809    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986812    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986816    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986819    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986821    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:23.136885 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986825    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986827    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986830    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986833    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986837    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986839    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986842    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986844    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986847    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986849    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986852    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986855    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986857    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986860    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986862    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986865    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.986867    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986931    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986943    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986949    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986953    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 14:56:23.137478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986957    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986960    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986967    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986971    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986974    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986977    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986980    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986983    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986988    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986991    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986994    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.986997    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987000    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987003    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987007    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987011    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987014    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987018    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987021    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987025    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987028    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987031    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987034    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987037    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 14:56:23.138160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987040    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987043    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987046    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987049    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987053    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987056    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987059    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987061    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987064    2556 flags.go:64] FLAG: --enable-server="true"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987067    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987072    2556 flags.go:64] FLAG: --event-burst="100"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987075    2556 flags.go:64] FLAG: --event-qps="50"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987078    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987080    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987083    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987087    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987102    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987106    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987109    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987112    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987115    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987118    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987121    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987125    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987128    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 14:56:23.138855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987131    2556 flags.go:64] FLAG: --feature-gates=""
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987135    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987138    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987140    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987143    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987146    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987149    2556 flags.go:64] FLAG: --help="false"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987152    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987155    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987158    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987161    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987165    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987168    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987171    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987174    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987176    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987179    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987182    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987185    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987188    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987191    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987193    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987197    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987201    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 14:56:23.139601 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987204    2556 flags.go:64] FLAG: --lock-file=""
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987207    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987210    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987213    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987218    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987222    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987227    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987230    2556 flags.go:64] FLAG: --logging-format="text"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987232    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987235    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987239    2556 flags.go:64] FLAG: --manifest-url=""
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987241    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987246    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987248    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987253    2556 flags.go:64] FLAG: --max-pods="110"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987256    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987259    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987262    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987265    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987268    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987270    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987273    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987281    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987284    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987287    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 14:56:23.140240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987290    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987293    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987298    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987301    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987304    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987306    2556 flags.go:64] FLAG: --port="10250"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987309    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987313    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0608066ea383b0d40"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987316    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987319    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987322    2556 flags.go:64] FLAG: --register-node="true"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987325    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987329    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987334    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987337    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987340    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987343    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987347    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987349    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987352    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987356    2556 flags.go:64] FLAG: --runonce="false"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987359    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987362    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987365    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987367    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 14:56:23.141172 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987370    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987373    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987376    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987379    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987390    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987394    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987397    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987399    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987402    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987405    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987408    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987413    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987416    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987419    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987424    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987426    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987429    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987432    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987435    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987438    2556 flags.go:64] FLAG: --v="2"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987443    2556 flags.go:64] FLAG: --version="false"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987448    2556 flags.go:64] FLAG: --vmodule=""
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987452    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.987455    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987565    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:23.144385 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987569    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987573    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987576    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987579    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987581    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987584    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987587    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987589    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987592    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987595    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987597    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987600    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987602    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987605    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987608    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987610    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987613    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987615    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987618    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987621    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:23.145474 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987623    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987627    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987629    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987633    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987635    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987638    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987641    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987645    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987649    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987653    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987656    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987658    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987661    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987663    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987666    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987669    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987671    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987674    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987676    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:23.152606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987679    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987682    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987684    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987686    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987689    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987692    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987694    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987697    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987700    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987702    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987704    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987707    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987710    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987713    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987718    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987721    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987724    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987727    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987730    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987732    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:23.239107 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987736    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987739    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987741    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987744    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987747    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987750    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987752    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987755    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987757    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987760    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987763    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987765    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987768    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987770    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987773    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987775    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987778    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987780    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987783    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987785    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:23.253651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987788    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987808    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987811    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987813    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987816    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987818    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987822    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987825    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987827    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987830    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987832    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987835    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987840    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987843    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987847    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987850    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987854    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987857    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:23.254298 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.987860    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.988471    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.995905    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.995920    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.995939    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.995944    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996000    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996006    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996009    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996013    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996015    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996018    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996021    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996024    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:23.255146 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996027    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996030    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996032    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996035    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996037    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996041    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996045    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996049    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996052    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996055    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996058    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996061    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996064    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996067    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996070    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996072    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996076    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996078    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996081    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996083    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:23.255785 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996086    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996089    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996092    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996094    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996097    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996100    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996103    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996105    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996108    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996110    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996113    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996116    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996119    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996122    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996125    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996127    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996130    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996133    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996136    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996139    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:23.256405 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996141    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996144    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996146    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996149    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996151    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996154    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996156    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996159    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996162    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996165    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996168    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996171    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996173    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996176    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996178    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996181    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996183    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996186    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996188    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996191    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:23.257029 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996194    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996196    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996199    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996204    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996207    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996209    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996212    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996215    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996217    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996220    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996223    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996225    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996228    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996231    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996234    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996236    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996239    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996241    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996244    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996246    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:23.257649 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996249    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996251    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996254    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996257    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996260    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996264    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996267    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996269    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996272    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996275    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996277    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.996282    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996402    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996406    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:23.258276 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996409    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996411    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996416    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996419    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996421    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996424    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996428    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996432    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996436    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996439    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996442    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996445    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996447    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996451    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996453    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996456    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996458    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996461    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996464    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:23.258687 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996466    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996469    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996472    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996474    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996477    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996479    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996482    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996484    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996487    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996490    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996493    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996496    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996499    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996502    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996504    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996508    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996511    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996513    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996516    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996518    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:23.259316 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996521    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996525    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996528    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996530    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996533    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996535    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996538    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996541    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996544    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996546    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996549    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996551    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996554    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996557    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996561    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996564    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996567    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996570    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996573    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:23.259942 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996575    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996578    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996580    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996583    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996585    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996588    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996590    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996593    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996597    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996599    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996602    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996604    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996607    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996610    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996615    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996618    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996620    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996623    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996625    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996628    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:23.260561 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996630    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996633    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996635    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996638    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996640    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996643    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996645    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996648    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996651    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996654    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996656    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996659    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996661    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996663    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996666    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996669    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996671    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996674    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:23.261188 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:22.996684    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.996690    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.996808    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:22.999779    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.001144    2556 server.go:1019] "Starting client certificate rotation"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.003856    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.007327    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.008130    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.009765    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.023482    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.025602    2556 log.go:25] "Validated CRI v1 image API"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.026487    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 14:56:23.262040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.028710    2556 fs.go:135] Filesystem UUIDs: map[1fef29a9-f83f-4d1b-8a19-601207c5b3e4:/dev/nvme0n1p4 213db6bb-904c-4317-9fb8-e9a5bc5bf310:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 07 14:56:23.262408 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.028724    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.033737    2556 manager.go:216] Machine: {Timestamp:2026-07-07 14:56:23.032345335 +0000 UTC m=+0.380827957 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100190 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29140ff5fc3c9adeba37e060999be1 SystemUUID:ec29140f-f5fc-3c9a-deba-37e060999be1 BootID:c977d40c-6eb1-403e-8f9d-9d2c1fbc4db9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:13:c2:b4:ef:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:13:c2:b4:ef:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:e5:a3:9c:76:61 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.033887    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.033976    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.037774    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.037822    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.038512    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.038521    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 14:56:23.262517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.038528    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.039312    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.040685    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.043682    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.043826    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.043854    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.043863    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.043873    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.043881    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.043889    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.044214    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.044230    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.045785    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.046830    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.048641    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.048668    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049663    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049680    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049689    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049697    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049709    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049716    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049728    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049735    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049744    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049752    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049769    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.049776    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.050820    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.050834    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.062958    2556 server.go:1268] "Started kubelet"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.063084    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.063402    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.063471    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.065015    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.066253    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.066324    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.070352    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.076753    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 14:56:23.262821 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.076927    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.076696    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-178.ec2.internal.18c009c26b0b3381  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-178.ec2.internal,UID:ip-10-0-134-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-178.ec2.internal,},FirstTimestamp:2026-07-07 14:56:23.062918017 +0000 UTC m=+0.411400640,LastTimestamp:2026-07-07 14:56:23.062918017 +0000 UTC m=+0.411400640,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-178.ec2.internal,}"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.082439    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.082511    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.082647    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.083057    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.083068    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.083067    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.083143    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.083150    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.084894    2556 factory.go:55] Registering systemd factory
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.084909    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.085115    2556 factory.go:153] Registering CRI-O factory
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.085130    2556 factory.go:223] Registration of the crio container factory successfully
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.085172    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.085191    2556 factory.go:103] Registering Raw factory
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.085200    2556 manager.go:1196] Started watching for new ooms in manager
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.085495    2556 manager.go:349] Starting recovery of all containers
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.088296    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.088850    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.090958    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.097955    2556 manager.go:354] Recovery completed
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.099154    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w87lf"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.102046    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.107179    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w87lf"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.107467    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.107494    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.107507    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.108020    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.108033    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.108051    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.108055    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.108699    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:23.264149 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.108726    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.108738    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.109117    2556 policy_none.go:49] "None policy: Start"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.109128    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.109137    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.109370    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-178.ec2.internal.18c009c26db32c44  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-178.ec2.internal,UID:ip-10-0-134-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-178.ec2.internal,},FirstTimestamp:2026-07-07 14:56:23.107480644 +0000 UTC m=+0.455963267,LastTimestamp:2026-07-07 14:56:23.107480644 +0000 UTC m=+0.455963267,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-178.ec2.internal,}"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.110368    2556 policy_none.go:47] "Start"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.143400    2556 manager.go:337] "Starting Device Plugin manager"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.143430    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.143443    2556 server.go:87] "Starting device plugin registration server"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.143645    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.143656    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.145808    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.145872    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.145880    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.146296    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.146339    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.243992    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.244897    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.244923    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.244932    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.244956    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.263630    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.263671    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.263693    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 14:56:23.265497 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.263732    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 14:56:23.352779 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.267851    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.352779 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.267868    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-178.ec2.internal\": node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.352779 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.268127    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:23.352779 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.337654    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.364714    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"]
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.364768    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.366307    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.366330    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.366341    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.367487    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.367614    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.367638    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.368182    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.368205    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.368214    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.368210    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.368292    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.368305    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.369478    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.369506    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.370084    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.370099    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.370112    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.384657    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90b7955341960021bd0b5ed66f077c6f-config\") pod \"kube-apiserver-proxy-ip-10-0-134-178.ec2.internal\" (UID: \"90b7955341960021bd0b5ed66f077c6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.384686    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7af05a3fbb064bc6b9824d1d785ba3dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"7af05a3fbb064bc6b9824d1d785ba3dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.384704    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7af05a3fbb064bc6b9824d1d785ba3dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"7af05a3fbb064bc6b9824d1d785ba3dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.395311    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-178.ec2.internal\" not found" node="ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.430024 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.400570    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-178.ec2.internal\" not found" node="ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.513707 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.438172    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.513707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.484859    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90b7955341960021bd0b5ed66f077c6f-config\") pod \"kube-apiserver-proxy-ip-10-0-134-178.ec2.internal\" (UID: \"90b7955341960021bd0b5ed66f077c6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.513707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.484882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7af05a3fbb064bc6b9824d1d785ba3dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"7af05a3fbb064bc6b9824d1d785ba3dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.513707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.484900    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7af05a3fbb064bc6b9824d1d785ba3dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"7af05a3fbb064bc6b9824d1d785ba3dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.513707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.484946    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7af05a3fbb064bc6b9824d1d785ba3dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"7af05a3fbb064bc6b9824d1d785ba3dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.513707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.484952    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90b7955341960021bd0b5ed66f077c6f-config\") pod \"kube-apiserver-proxy-ip-10-0-134-178.ec2.internal\" (UID: \"90b7955341960021bd0b5ed66f077c6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.513707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.484973    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7af05a3fbb064bc6b9824d1d785ba3dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"7af05a3fbb064bc6b9824d1d785ba3dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.622096 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.538962    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.726156 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.642681    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.726156 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.698048    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.726156 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.704111    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:23.831834 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.742906    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.957214 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:23.843479    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 07 14:56:23.957214 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.910949    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:23.957214 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.954872    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.983913    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:23.999701    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.001483    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.008514    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.008608    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.008649    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.008667    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.008675    2556 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a14015b673e2c413db070b5bb8999aa7-39028a6a2a7620f4.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.134.178:38078->100.57.72.217:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.046516    2556 apiserver.go:52] "Watching apiserver"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.072791    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.073136    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd","openshift-cluster-node-tuning-operator/tuned-x5xvg","openshift-image-registry/node-ca-9q9q5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal","openshift-multus/multus-additional-cni-plugins-n6gbg","openshift-multus/network-metrics-daemon-wxd9k","kube-system/global-pull-secret-syncer-f7mm6","kube-system/konnectivity-agent-bpb5c","openshift-dns/node-resolver-x4xx9","openshift-multus/multus-nf625","openshift-network-diagnostics/network-check-target-756dp","openshift-network-operator/iptables-alerter-d9rmr","openshift-ovn-kubernetes/ovnkube-node-ssnrw"]
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.074507    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.076534    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.078827    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.079453    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:24.079558 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.079512    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:24.081790 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.080606    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.084495 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.081817    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.084495 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.082875    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 14:56:24.084495 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.083012    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.084495 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.084397    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.085741 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085552    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 14:56:24.085741 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085565    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zf26w\""
Jul 07 14:56:24.085741 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085569    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085782    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085790    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085813    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085922    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6wj7v\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085942    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085949    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w9sqg\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.085983    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086085    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086096    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086085    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 14:56:24.086335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086249    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 14:56:24.086987 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086375    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.086987 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.086430    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:24.086987 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086509    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bml5m\""
Jul 07 14:56:24.086987 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086646    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 14:56:24.086987 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086710    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 14:56:24.086987 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086852    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5dwqx\""
Jul 07 14:56:24.086987 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.086921    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087692    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-log-socket\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087758    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-kubernetes\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087801    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-conf-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087825    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087829    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-multus-certs\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.087891    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087904    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-systemd\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087921    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087954    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d444a611-792d-4ff0-9cca-cda0856d9577-ovn-node-metrics-cert\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.087985    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b19a2c4-299a-4d9e-a470-50b1d292899f-hosts-file\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088045    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfwpc\" (UniqueName: \"kubernetes.io/projected/0b19a2c4-299a-4d9e-a470-50b1d292899f-kube-api-access-mfwpc\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088070    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-tuned\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.088233 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088093    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-system-cni-dir\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088111    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c835e94b-6053-4750-b83d-354e135e1e8f-agent-certs\") pod \"konnectivity-agent-bpb5c\" (UID: \"c835e94b-6053-4750-b83d-354e135e1e8f\") " pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088126    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-run-ovn-kubernetes\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-modprobe-d\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088162    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-run\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088178    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z57sp\" (UniqueName: \"kubernetes.io/projected/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-kube-api-access-z57sp\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088194    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2psb\" (UniqueName: \"kubernetes.io/projected/91074283-7c9c-4a20-9196-2d55a7bc6ae1-kube-api-access-g2psb\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088209    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-kubelet\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088228    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-env-overrides\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088249    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysctl-d\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088265    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-system-cni-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088282    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7246a8b1-2ce5-412f-925b-27e83323ed25-cni-binary-copy\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088297    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088329    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b19a2c4-299a-4d9e-a470-50b1d292899f-tmp-dir\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.200831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088354    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-host\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088371    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-netns\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-hostroot\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088428    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-967kp\" (UniqueName: \"kubernetes.io/projected/8dfa10f2-79ee-4bca-8ba6-65fdce870661-kube-api-access-967kp\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088447    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-os-release\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088463    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-socket-dir-parent\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-cni-multus\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088492    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-systemd-units\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088506    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-systemd\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088520    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-cni-bin\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088543    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysconfig\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088560    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-lib-modules\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088580    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-cni-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088595    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.201471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088617    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-slash\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088641    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088661    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-ovnkube-config\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088682    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-sys\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088706    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-etc-kubernetes\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-ovnkube-script-lib\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088745    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91074283-7c9c-4a20-9196-2d55a7bc6ae1-host\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088760    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c835e94b-6053-4750-b83d-354e135e1e8f-konnectivity-ca\") pod \"konnectivity-agent-bpb5c\" (UID: \"c835e94b-6053-4750-b83d-354e135e1e8f\") " pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088774    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-run-netns\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088817    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-var-lib-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088837    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-cni-netd\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088859    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-cni-bin\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088877    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-os-release\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088912    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-ovn\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088938    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-daemon-config\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088968    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.088986    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91074283-7c9c-4a20-9196-2d55a7bc6ae1-serviceca\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089000    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-etc-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089015    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-node-log\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089038    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf4vh\" (UniqueName: \"kubernetes.io/projected/d444a611-792d-4ff0-9cca-cda0856d9577-kube-api-access-xf4vh\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089065    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089090    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-var-lib-kubelet\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089108    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-kubelet\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cnibin\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysctl-conf\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089159    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dfa10f2-79ee-4bca-8ba6-65fdce870661-tmp\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089176    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-cnibin\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089202    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-k8s-cni-cncf-io\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.202846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089229    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjd54\" (UniqueName: \"kubernetes.io/projected/7246a8b1-2ce5-412f-925b-27e83323ed25-kube-api-access-kjd54\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089320    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089577    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089693    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089700    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.089949    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.090573    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.091808    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.098749    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.098765    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jzhpl\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.098754    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099051    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099067    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099179    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099179    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z2mgc\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099246    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099278    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zxnk7\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099278    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v9s2t\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099326    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.099502    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.109371    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:51:23 +0000 UTC" deadline="2028-03-04 17:09:32.597859078 +0000 UTC"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.109412    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14546h13m8.488449537s"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.120167    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.143922    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d4864"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.159570    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d4864"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.184100    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189532    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-host\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189557    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-netns\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.203561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189576    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-hostroot\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189596    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k9vx\" (UniqueName: \"kubernetes.io/projected/b6fd483f-453e-4289-82fb-02ad4d241017-kube-api-access-4k9vx\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189620    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-device-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189647    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-967kp\" (UniqueName: \"kubernetes.io/projected/8dfa10f2-79ee-4bca-8ba6-65fdce870661-kube-api-access-967kp\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189656    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-hostroot\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189675    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-os-release\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189646    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-netns\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189699    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-socket-dir-parent\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189695    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-host\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189728    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-cni-multus\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189781    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-cni-multus\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189807    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-os-release\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189790    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-systemd-units\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189842    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-socket-dir-parent\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.204549 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189858    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-systemd-units\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189880    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-systemd\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189906    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-cni-bin\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189932    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dec33f92-47f6-4040-8a3b-4658fdbf72ca-dbus\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysconfig\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189967    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-lib-modules\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189930    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-systemd\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.189996    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-cni-bin\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190024    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-cni-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190040    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysconfig\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190050    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190072    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-lib-modules\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-slash\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190109    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-host-slash\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.207410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190134    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190153    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-slash\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190166    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-ovnkube-config\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190176    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-cni-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190193    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190219    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-sys\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-etc-kubernetes\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190274    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-sys\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190296    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-ovnkube-script-lib\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190316    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-etc-kubernetes\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190321    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190349    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-iptables-alerter-script\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190374    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-registration-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91074283-7c9c-4a20-9196-2d55a7bc6ae1-host\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.208057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190424    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c835e94b-6053-4750-b83d-354e135e1e8f-konnectivity-ca\") pod \"konnectivity-agent-bpb5c\" (UID: \"c835e94b-6053-4750-b83d-354e135e1e8f\") " pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190443    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-run-netns\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190458    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-var-lib-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190452    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91074283-7c9c-4a20-9196-2d55a7bc6ae1-host\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190487    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-run-netns\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190493    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-var-lib-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190511    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-cni-netd\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190532    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqx4s\" (UniqueName: \"kubernetes.io/projected/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-kube-api-access-nqx4s\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-cni-bin\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190563    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-cni-netd\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190584    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-os-release\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190601    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-ovn\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190672    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-ovn\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190676    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-cni-bin\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.210676 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190675    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-os-release\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190695    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-sys-fs\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-daemon-config\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190770    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91074283-7c9c-4a20-9196-2d55a7bc6ae1-serviceca\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190828    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-etc-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-node-log\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4vh\" (UniqueName: \"kubernetes.io/projected/d444a611-792d-4ff0-9cca-cda0856d9577-kube-api-access-xf4vh\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190926    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-var-lib-kubelet\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190948    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-kubelet\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.190972    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cnibin\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191009    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-socket-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191033    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysctl-conf\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.211485 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191057    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dfa10f2-79ee-4bca-8ba6-65fdce870661-tmp\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191079    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-cnibin\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191102    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-k8s-cni-cncf-io\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjd54\" (UniqueName: \"kubernetes.io/projected/7246a8b1-2ce5-412f-925b-27e83323ed25-kube-api-access-kjd54\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191158    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191174    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-etc-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-log-socket\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-kubernetes\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191221    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-node-log\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191239    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-conf-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191264    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-multus-certs\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191286    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-systemd\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191311    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191337    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d444a611-792d-4ff0-9cca-cda0856d9577-ovn-node-metrics-cert\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191351    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysctl-conf\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.212197 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b19a2c4-299a-4d9e-a470-50b1d292899f-hosts-file\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191386    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfwpc\" (UniqueName: \"kubernetes.io/projected/0b19a2c4-299a-4d9e-a470-50b1d292899f-kube-api-access-mfwpc\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191408    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-tuned\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191492    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-system-cni-dir\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191517    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c835e94b-6053-4750-b83d-354e135e1e8f-agent-certs\") pod \"konnectivity-agent-bpb5c\" (UID: \"c835e94b-6053-4750-b83d-354e135e1e8f\") " pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-run-ovn-kubernetes\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191592    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-var-lib-kubelet\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191638    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8cmm\" (UniqueName: \"kubernetes.io/projected/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-kube-api-access-c8cmm\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191644    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91074283-7c9c-4a20-9196-2d55a7bc6ae1-serviceca\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191650    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-var-lib-kubelet\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191703    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-modprobe-d\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191728    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-run\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191750    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-ovnkube-config\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191839    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.319054 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191897    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-ovnkube-script-lib\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191752    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z57sp\" (UniqueName: \"kubernetes.io/projected/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-kube-api-access-z57sp\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191965    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cnibin\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.191985    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2psb\" (UniqueName: \"kubernetes.io/projected/91074283-7c9c-4a20-9196-2d55a7bc6ae1-kube-api-access-g2psb\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192007    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-kubelet\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192040    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-kubelet\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192083    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192257    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-daemon-config\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192319    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-log-socket\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192372    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-cnibin\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192412    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-k8s-cni-cncf-io\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192471    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-openvswitch\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192524    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-run-ovn-kubernetes\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192574    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b19a2c4-299a-4d9e-a470-50b1d292899f-hosts-file\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192591    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c835e94b-6053-4750-b83d-354e135e1e8f-konnectivity-ca\") pod \"konnectivity-agent-bpb5c\" (UID: \"c835e94b-6053-4750-b83d-354e135e1e8f\") " pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.319978 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192612    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-modprobe-d\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192663    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-system-cni-dir\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192669    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-run\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192778    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-env-overrides\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192825    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192898    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-run-systemd\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192897    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192939    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-host-run-multus-certs\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192955    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-kubernetes\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192975    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-multus-conf-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.192988    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dec33f92-47f6-4040-8a3b-4658fdbf72ca-kubelet-config\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193012    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d444a611-792d-4ff0-9cca-cda0856d9577-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193022    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysctl-d\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193057    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-system-cni-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.320602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193100    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-sysctl-d\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193140    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7246a8b1-2ce5-412f-925b-27e83323ed25-cni-binary-copy\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193166    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193203    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7246a8b1-2ce5-412f-925b-27e83323ed25-system-cni-dir\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193215    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b19a2c4-299a-4d9e-a470-50b1d292899f-tmp-dir\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193472    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d444a611-792d-4ff0-9cca-cda0856d9577-env-overrides\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193636    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b19a2c4-299a-4d9e-a470-50b1d292899f-tmp-dir\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.193979    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.194243    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7246a8b1-2ce5-412f-925b-27e83323ed25-cni-binary-copy\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.195733    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d444a611-792d-4ff0-9cca-cda0856d9577-ovn-node-metrics-cert\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.195785    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dfa10f2-79ee-4bca-8ba6-65fdce870661-tmp\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.195939    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c835e94b-6053-4750-b83d-354e135e1e8f-agent-certs\") pod \"konnectivity-agent-bpb5c\" (UID: \"c835e94b-6053-4750-b83d-354e135e1e8f\") " pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.195963    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8dfa10f2-79ee-4bca-8ba6-65fdce870661-etc-tuned\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.210874    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.210898    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:24.322406 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.210912    2556 projected.go:196] Error preparing data for projected volume kube-api-access-wv4c5 for pod openshift-network-diagnostics/network-check-target-756dp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.210992    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5 podName:70153084-d026-4d8c-a91c-dff91ff1f515 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:24.710972545 +0000 UTC m=+2.059455165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wv4c5" (UniqueName: "kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5") pod "network-check-target-756dp" (UID: "70153084-d026-4d8c-a91c-dff91ff1f515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.213511    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-967kp\" (UniqueName: \"kubernetes.io/projected/8dfa10f2-79ee-4bca-8ba6-65fdce870661-kube-api-access-967kp\") pod \"tuned-x5xvg\" (UID: \"8dfa10f2-79ee-4bca-8ba6-65fdce870661\") " pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.213545    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z57sp\" (UniqueName: \"kubernetes.io/projected/c0fc80cb-6c79-482c-b8f3-d6856ea7d72c-kube-api-access-z57sp\") pod \"multus-additional-cni-plugins-n6gbg\" (UID: \"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c\") " pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.213644    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfwpc\" (UniqueName: \"kubernetes.io/projected/0b19a2c4-299a-4d9e-a470-50b1d292899f-kube-api-access-mfwpc\") pod \"node-resolver-x4xx9\" (UID: \"0b19a2c4-299a-4d9e-a470-50b1d292899f\") " pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.213704    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf4vh\" (UniqueName: \"kubernetes.io/projected/d444a611-792d-4ff0-9cca-cda0856d9577-kube-api-access-xf4vh\") pod \"ovnkube-node-ssnrw\" (UID: \"d444a611-792d-4ff0-9cca-cda0856d9577\") " pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.213711    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2psb\" (UniqueName: \"kubernetes.io/projected/91074283-7c9c-4a20-9196-2d55a7bc6ae1-kube-api-access-g2psb\") pod \"node-ca-9q9q5\" (UID: \"91074283-7c9c-4a20-9196-2d55a7bc6ae1\") " pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.213995    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjd54\" (UniqueName: \"kubernetes.io/projected/7246a8b1-2ce5-412f-925b-27e83323ed25-kube-api-access-kjd54\") pod \"multus-nf625\" (UID: \"7246a8b1-2ce5-412f-925b-27e83323ed25\") " pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.256551    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x4xx9"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.277439    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9q9q5"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293432    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293460    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dec33f92-47f6-4040-8a3b-4658fdbf72ca-dbus\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293488    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-host-slash\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293505    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.323082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293559    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-host-slash\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293559    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dec33f92-47f6-4040-8a3b-4658fdbf72ca-dbus\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293583    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293591    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293613    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-iptables-alerter-script\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293643    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-registration-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.293644    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqx4s\" (UniqueName: \"kubernetes.io/projected/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-kube-api-access-nqx4s\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.293690    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293694    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-sys-fs\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293719    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-registration-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293756    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-sys-fs\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.293769    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret podName:dec33f92-47f6-4040-8a3b-4658fdbf72ca nodeName:}" failed. No retries permitted until 2026-07-07 14:56:24.79374382 +0000 UTC m=+2.142226454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret") pod "global-pull-secret-syncer-f7mm6" (UID: "dec33f92-47f6-4040-8a3b-4658fdbf72ca") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:24.325492 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.293810    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:24.793796436 +0000 UTC m=+2.142279045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-socket-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293937    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8cmm\" (UniqueName: \"kubernetes.io/projected/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-kube-api-access-c8cmm\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.293966    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-socket-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dec33f92-47f6-4040-8a3b-4658fdbf72ca-kubelet-config\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294067    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294089    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k9vx\" (UniqueName: \"kubernetes.io/projected/b6fd483f-453e-4289-82fb-02ad4d241017-kube-api-access-4k9vx\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294114    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-device-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294189    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-device-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294241    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dec33f92-47f6-4040-8a3b-4658fdbf72ca-kubelet-config\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294295    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.294326    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-iptables-alerter-script\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.312083    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k9vx\" (UniqueName: \"kubernetes.io/projected/b6fd483f-453e-4289-82fb-02ad4d241017-kube-api-access-4k9vx\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.312552    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqx4s\" (UniqueName: \"kubernetes.io/projected/cb5585b7-b224-4b7b-b61b-ccdd7c3165b9-kube-api-access-nqx4s\") pod \"iptables-alerter-d9rmr\" (UID: \"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9\") " pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.325971 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.312566    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8cmm\" (UniqueName: \"kubernetes.io/projected/cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1-kube-api-access-c8cmm\") pod \"aws-ebs-csi-driver-node-jlcdd\" (UID: \"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.430380 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.314543    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:24.430380 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.424779    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:24.458940 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.458863    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:24.507258 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.465390    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x5xvg"
Jul 07 14:56:24.507668 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.507638    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6gbg"
Jul 07 14:56:24.507775 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.507680    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nf625"
Jul 07 14:56:24.565897 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.565850    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af05a3fbb064bc6b9824d1d785ba3dd.slice/crio-643c0a8be8e52553dd7a7ca65d77c48cab863b4a62dfb0b73b66429d108ca1fe WatchSource:0}: Error finding container 643c0a8be8e52553dd7a7ca65d77c48cab863b4a62dfb0b73b66429d108ca1fe: Status 404 returned error can't find the container with id 643c0a8be8e52553dd7a7ca65d77c48cab863b4a62dfb0b73b66429d108ca1fe
Jul 07 14:56:24.566268 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.566219    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b7955341960021bd0b5ed66f077c6f.slice/crio-eab36db27c09a78ecfdf871a820fab00aa8416f7d11d5f24bc7d804c33e6d8eb WatchSource:0}: Error finding container eab36db27c09a78ecfdf871a820fab00aa8416f7d11d5f24bc7d804c33e6d8eb: Status 404 returned error can't find the container with id eab36db27c09a78ecfdf871a820fab00aa8416f7d11d5f24bc7d804c33e6d8eb
Jul 07 14:56:24.574332 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.574314    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:56:24.583354 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.583326    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b19a2c4_299a_4d9e_a470_50b1d292899f.slice/crio-fe8c889515be3f4c23effa18fb9d8f1da8e45c36d0b39ce96d83622dee3f4a11 WatchSource:0}: Error finding container fe8c889515be3f4c23effa18fb9d8f1da8e45c36d0b39ce96d83622dee3f4a11: Status 404 returned error can't find the container with id fe8c889515be3f4c23effa18fb9d8f1da8e45c36d0b39ce96d83622dee3f4a11
Jul 07 14:56:24.599340 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.599319    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d9rmr"
Jul 07 14:56:24.604153 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.604133    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd"
Jul 07 14:56:24.614245 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.614217    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5585b7_b224_4b7b_b61b_ccdd7c3165b9.slice/crio-9e77971c5d8bfb83dbc8d2ef0d0b629cd514d1f1653c5dd6399a070a9d687f90 WatchSource:0}: Error finding container 9e77971c5d8bfb83dbc8d2ef0d0b629cd514d1f1653c5dd6399a070a9d687f90: Status 404 returned error can't find the container with id 9e77971c5d8bfb83dbc8d2ef0d0b629cd514d1f1653c5dd6399a070a9d687f90
Jul 07 14:56:24.614936 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.614905    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1c7fbf_4dc9_4066_b7ec_c2bea75935a1.slice/crio-3ddd21fbe50696c734cc1a08d62eddeff58ccaa8c662849475734c1f0450d951 WatchSource:0}: Error finding container 3ddd21fbe50696c734cc1a08d62eddeff58ccaa8c662849475734c1f0450d951: Status 404 returned error can't find the container with id 3ddd21fbe50696c734cc1a08d62eddeff58ccaa8c662849475734c1f0450d951
Jul 07 14:56:24.616030 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.615982    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91074283_7c9c_4a20_9196_2d55a7bc6ae1.slice/crio-e949af3ed63a52ba5bf60a39081420863b07531755168493fe493658c5dd2b18 WatchSource:0}: Error finding container e949af3ed63a52ba5bf60a39081420863b07531755168493fe493658c5dd2b18: Status 404 returned error can't find the container with id e949af3ed63a52ba5bf60a39081420863b07531755168493fe493658c5dd2b18
Jul 07 14:56:24.799943 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.799913    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc835e94b_6053_4750_b83d_354e135e1e8f.slice/crio-e1e59713a614244c4562554dd5ab31fc42876fbfbba314d064ef78766667611e WatchSource:0}: Error finding container e1e59713a614244c4562554dd5ab31fc42876fbfbba314d064ef78766667611e: Status 404 returned error can't find the container with id e1e59713a614244c4562554dd5ab31fc42876fbfbba314d064ef78766667611e
Jul 07 14:56:24.810632 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.810607    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:24.810727 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.810654    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:24.810767 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810745    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:24.810801 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810766    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:24.810801 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810789    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret podName:dec33f92-47f6-4040-8a3b-4658fdbf72ca nodeName:}" failed. No retries permitted until 2026-07-07 14:56:25.810776119 +0000 UTC m=+3.159258730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret") pod "global-pull-secret-syncer-f7mm6" (UID: "dec33f92-47f6-4040-8a3b-4658fdbf72ca") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:24.810801 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810791    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:24.810929 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810823    2556 projected.go:196] Error preparing data for projected volume kube-api-access-wv4c5 for pod openshift-network-diagnostics/network-check-target-756dp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:24.810929 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810857    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:24.810929 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.810783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:24.810929 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810862    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5 podName:70153084-d026-4d8c-a91c-dff91ff1f515 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:25.810850227 +0000 UTC m=+3.159332837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wv4c5" (UniqueName: "kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5") pod "network-check-target-756dp" (UID: "70153084-d026-4d8c-a91c-dff91ff1f515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:24.810929 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:24.810915    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:25.810903505 +0000 UTC m=+3.159386114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:24.861828 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.861782    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7246a8b1_2ce5_412f_925b_27e83323ed25.slice/crio-fdbde571576bd882791fba9753d82529247e68f17fdfd7f844d2892f2b673c85 WatchSource:0}: Error finding container fdbde571576bd882791fba9753d82529247e68f17fdfd7f844d2892f2b673c85: Status 404 returned error can't find the container with id fdbde571576bd882791fba9753d82529247e68f17fdfd7f844d2892f2b673c85
Jul 07 14:56:24.885730 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:24.885705    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:24.997674 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:24.997640    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd444a611_792d_4ff0_9cca_cda0856d9577.slice/crio-94489873b6f367ad36af7b2b8b4b475c1cc939590aba9ec418c26d2af8f41640 WatchSource:0}: Error finding container 94489873b6f367ad36af7b2b8b4b475c1cc939590aba9ec418c26d2af8f41640: Status 404 returned error can't find the container with id 94489873b6f367ad36af7b2b8b4b475c1cc939590aba9ec418c26d2af8f41640
Jul 07 14:56:25.001498 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:25.001470    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fc80cb_6c79_482c_b8f3_d6856ea7d72c.slice/crio-ecfea970fccec019d5e9742dd1540d3d2cd8497fe48d48aa5021e8f16f8482a3 WatchSource:0}: Error finding container ecfea970fccec019d5e9742dd1540d3d2cd8497fe48d48aa5021e8f16f8482a3: Status 404 returned error can't find the container with id ecfea970fccec019d5e9742dd1540d3d2cd8497fe48d48aa5021e8f16f8482a3
Jul 07 14:56:25.001848 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:25.001830    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfa10f2_79ee_4bca_8ba6_65fdce870661.slice/crio-7a36b0f8c6aa270b26a2225ea689485e789fa4b2568d18b27be13fa853779831 WatchSource:0}: Error finding container 7a36b0f8c6aa270b26a2225ea689485e789fa4b2568d18b27be13fa853779831: Status 404 returned error can't find the container with id 7a36b0f8c6aa270b26a2225ea689485e789fa4b2568d18b27be13fa853779831
Jul 07 14:56:25.162469 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.162357    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:51:24 +0000 UTC" deadline="2028-02-23 14:01:45.498709253 +0000 UTC"
Jul 07 14:56:25.162469 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.162408    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14303h5m20.336303971s"
Jul 07 14:56:25.264787 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.264253    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:25.264787 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.264406    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:25.270374 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.270319    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerStarted","Data":"ecfea970fccec019d5e9742dd1540d3d2cd8497fe48d48aa5021e8f16f8482a3"}
Jul 07 14:56:25.271692 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.271637    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9q9q5" event={"ID":"91074283-7c9c-4a20-9196-2d55a7bc6ae1","Type":"ContainerStarted","Data":"e949af3ed63a52ba5bf60a39081420863b07531755168493fe493658c5dd2b18"}
Jul 07 14:56:25.274411 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.274336    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd" event={"ID":"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1","Type":"ContainerStarted","Data":"3ddd21fbe50696c734cc1a08d62eddeff58ccaa8c662849475734c1f0450d951"}
Jul 07 14:56:25.276187 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.276132    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x4xx9" event={"ID":"0b19a2c4-299a-4d9e-a470-50b1d292899f","Type":"ContainerStarted","Data":"fe8c889515be3f4c23effa18fb9d8f1da8e45c36d0b39ce96d83622dee3f4a11"}
Jul 07 14:56:25.277681 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.277656    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x5xvg" event={"ID":"8dfa10f2-79ee-4bca-8ba6-65fdce870661","Type":"ContainerStarted","Data":"7a36b0f8c6aa270b26a2225ea689485e789fa4b2568d18b27be13fa853779831"}
Jul 07 14:56:25.279184 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.279158    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"94489873b6f367ad36af7b2b8b4b475c1cc939590aba9ec418c26d2af8f41640"}
Jul 07 14:56:25.281290 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.281249    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nf625" event={"ID":"7246a8b1-2ce5-412f-925b-27e83323ed25","Type":"ContainerStarted","Data":"fdbde571576bd882791fba9753d82529247e68f17fdfd7f844d2892f2b673c85"}
Jul 07 14:56:25.282997 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.282971    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bpb5c" event={"ID":"c835e94b-6053-4750-b83d-354e135e1e8f","Type":"ContainerStarted","Data":"e1e59713a614244c4562554dd5ab31fc42876fbfbba314d064ef78766667611e"}
Jul 07 14:56:25.285324 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.285299    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d9rmr" event={"ID":"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9","Type":"ContainerStarted","Data":"9e77971c5d8bfb83dbc8d2ef0d0b629cd514d1f1653c5dd6399a070a9d687f90"}
Jul 07 14:56:25.286673 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.286648    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" event={"ID":"7af05a3fbb064bc6b9824d1d785ba3dd","Type":"ContainerStarted","Data":"643c0a8be8e52553dd7a7ca65d77c48cab863b4a62dfb0b73b66429d108ca1fe"}
Jul 07 14:56:25.288202 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.288179    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal" event={"ID":"90b7955341960021bd0b5ed66f077c6f","Type":"ContainerStarted","Data":"eab36db27c09a78ecfdf871a820fab00aa8416f7d11d5f24bc7d804c33e6d8eb"}
Jul 07 14:56:25.819254 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.819218    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:25.819456 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.819282    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:25.819456 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:25.819344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:25.819577 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.819485    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:25.819577 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.819546    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret podName:dec33f92-47f6-4040-8a3b-4658fdbf72ca nodeName:}" failed. No retries permitted until 2026-07-07 14:56:27.81952655 +0000 UTC m=+5.168009168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret") pod "global-pull-secret-syncer-f7mm6" (UID: "dec33f92-47f6-4040-8a3b-4658fdbf72ca") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:25.820026 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.819996    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:25.820026 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.820026    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:25.820184 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.820039    2556 projected.go:196] Error preparing data for projected volume kube-api-access-wv4c5 for pod openshift-network-diagnostics/network-check-target-756dp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:25.820184 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.820088    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5 podName:70153084-d026-4d8c-a91c-dff91ff1f515 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:27.820071848 +0000 UTC m=+5.168554464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wv4c5" (UniqueName: "kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5") pod "network-check-target-756dp" (UID: "70153084-d026-4d8c-a91c-dff91ff1f515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:25.820184 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.820151    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:25.820184 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:25.820185    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:27.820174565 +0000 UTC m=+5.168657178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:26.163587 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:26.163364    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:51:24 +0000 UTC" deadline="2028-02-24 17:46:24.494350962 +0000 UTC"
Jul 07 14:56:26.163587 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:26.163401    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14330h49m58.330955387s"
Jul 07 14:56:26.264284 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:26.264254    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:26.264284 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:26.264272    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:26.264518 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:26.264391    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:26.264583 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:26.264540    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:26.562670 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:26.562592    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:27.091939 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:27.091859    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:27.267521 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:27.267493    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:27.267993 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.267639    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:27.837076 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:27.836973    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:27.837076 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:27.837048    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:27.837348 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:27.837094    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:27.837348 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837271    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:27.837348 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837285    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:27.837348 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837330    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:27.837348 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837345    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:31.837325997 +0000 UTC m=+9.185808613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:27.837348 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837350    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:27.837679 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837364    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret podName:dec33f92-47f6-4040-8a3b-4658fdbf72ca nodeName:}" failed. No retries permitted until 2026-07-07 14:56:31.837354065 +0000 UTC m=+9.185836680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret") pod "global-pull-secret-syncer-f7mm6" (UID: "dec33f92-47f6-4040-8a3b-4658fdbf72ca") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:27.837679 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837366    2556 projected.go:196] Error preparing data for projected volume kube-api-access-wv4c5 for pod openshift-network-diagnostics/network-check-target-756dp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:27.837679 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:27.837404    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5 podName:70153084-d026-4d8c-a91c-dff91ff1f515 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:31.837393466 +0000 UTC m=+9.185876088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wv4c5" (UniqueName: "kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5") pod "network-check-target-756dp" (UID: "70153084-d026-4d8c-a91c-dff91ff1f515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:28.264980 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:28.264414    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:28.264980 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:28.264547    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:28.264980 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:28.264580    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:28.264980 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:28.264740    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:29.264900 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:29.264349    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:29.264900 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:29.264513    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:30.264564 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:30.264533    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:30.264729 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:30.264655    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:30.265065 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:30.265043    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:30.265453 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:30.265138    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:31.264660 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:31.264146    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:31.264660 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.264285    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:31.870464 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:31.870434    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:31.870497    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:31.870543    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870627    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870658    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870674    2556 projected.go:196] Error preparing data for projected volume kube-api-access-wv4c5 for pod openshift-network-diagnostics/network-check-target-756dp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870686    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870688    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870727    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5 podName:70153084-d026-4d8c-a91c-dff91ff1f515 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:39.870709507 +0000 UTC m=+17.219192134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wv4c5" (UniqueName: "kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5") pod "network-check-target-756dp" (UID: "70153084-d026-4d8c-a91c-dff91ff1f515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870748    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:39.870737547 +0000 UTC m=+17.219220156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:31.870943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:31.870763    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret podName:dec33f92-47f6-4040-8a3b-4658fdbf72ca nodeName:}" failed. No retries permitted until 2026-07-07 14:56:39.870754969 +0000 UTC m=+17.219237579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret") pod "global-pull-secret-syncer-f7mm6" (UID: "dec33f92-47f6-4040-8a3b-4658fdbf72ca") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:32.264670 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:32.264599    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:32.264824 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:32.264705    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:32.265029 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:32.264601    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:32.265125 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:32.265101    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:33.265097 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:33.265021    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:33.265553 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:33.265151    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:34.264591 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:34.263948    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:34.264591 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:34.264069    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:34.264591 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:34.264420    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:34.264591 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:34.264509    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:35.266020 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:35.265484    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:35.266020 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:35.265657    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:36.263958 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:36.263919    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:36.264113 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:36.264021    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:36.264113 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:36.264046    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:36.264181 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:36.264140    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:37.264325 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:37.264295    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:37.264785 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:37.264432    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:38.264243 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:38.264207    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:38.264419 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:38.264249    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:38.264419 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:38.264329    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:38.264419 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:38.264397    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:39.263943 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:39.263905    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:39.264103 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.264026    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:39.934119 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:39.934090    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:39.934136    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:39.934171    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934237    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934286    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934303    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret podName:dec33f92-47f6-4040-8a3b-4658fdbf72ca nodeName:}" failed. No retries permitted until 2026-07-07 14:56:55.934283637 +0000 UTC m=+33.282766262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret") pod "global-pull-secret-syncer-f7mm6" (UID: "dec33f92-47f6-4040-8a3b-4658fdbf72ca") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934324    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:55.934313697 +0000 UTC m=+33.282796316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934290    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934351    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934364    2556 projected.go:196] Error preparing data for projected volume kube-api-access-wv4c5 for pod openshift-network-diagnostics/network-check-target-756dp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:39.934550 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:39.934400    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5 podName:70153084-d026-4d8c-a91c-dff91ff1f515 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:55.93439042 +0000 UTC m=+33.282873054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wv4c5" (UniqueName: "kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5") pod "network-check-target-756dp" (UID: "70153084-d026-4d8c-a91c-dff91ff1f515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:40.264449 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:40.264379    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:40.264597 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:40.264379    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:40.264597 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:40.264487    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:40.264686 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:40.264588    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:41.264156 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:41.264123    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:41.264562 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:41.264245    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:42.264648 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:42.264617    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:42.265088 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:42.264617    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:42.265088 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:42.264726    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:42.265088 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:42.264778    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:43.264687 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:43.264651    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:43.265161 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:43.264766    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:44.264005 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:44.263977    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:44.264151 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:44.263977    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:44.264151 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:44.264088    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:44.264265 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:44.264156    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:45.265188 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.264988    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:45.265519 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:45.265311    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:45.337774 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.337739    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"bdcba7a08a3978394863e03853be406951624e8a0a71d079a6d5b97824011e0f"}
Jul 07 14:56:45.337875 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.337824    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"cbd665a288b25ca2ff324b2bf6753454e7ae711c2803765cae0b75444f9c25ae"}
Jul 07 14:56:45.339218 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.339194    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nf625" event={"ID":"7246a8b1-2ce5-412f-925b-27e83323ed25","Type":"ContainerStarted","Data":"f90b3e22f3722c87e219217d37422dd5b805e42a786f976848da259e83b6738c"}
Jul 07 14:56:45.340652 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.340627    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal" event={"ID":"90b7955341960021bd0b5ed66f077c6f","Type":"ContainerStarted","Data":"28c604449fbb6b7b3eb49ba37ae2b147ea16a69237a74e2523b3260ccf676e54"}
Jul 07 14:56:45.341990 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.341969    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x5xvg" event={"ID":"8dfa10f2-79ee-4bca-8ba6-65fdce870661","Type":"ContainerStarted","Data":"ff636f3185378f40d0888c6418a17bfd4180383d48387e6199aa43fd5e51e0e8"}
Jul 07 14:56:45.361645 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.361603    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nf625" podStartSLOduration=2.30048426 podStartE2EDuration="22.361588863s" totalImagesPullingTime="20.061104603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:45.361491615 +0000 UTC m=+22.709974246" watchObservedRunningTime="2026-07-07 14:56:45.361588863 +0000 UTC m=+22.710071495"
Jul 07 14:56:45.381216 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.381110    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x5xvg" podStartSLOduration=2.4758385130000002 podStartE2EDuration="22.381098235s" totalImagesPullingTime="19.905259722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:45.380754398 +0000 UTC m=+22.729237048" watchObservedRunningTime="2026-07-07 14:56:45.381098235 +0000 UTC m=+22.729580865"
Jul 07 14:56:45.397981 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:45.397950    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal" podStartSLOduration=21.397939074 podStartE2EDuration="21.397939074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:45.397681331 +0000 UTC m=+22.746163961" watchObservedRunningTime="2026-07-07 14:56:45.397939074 +0000 UTC m=+22.746421704"
Jul 07 14:56:46.264371 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.264099    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:46.264578 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.264129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:46.264578 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:46.264456    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:46.264578 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:46.264537    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:46.346933 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.346913    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 14:56:46.347474 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.347251    2556 generic.go:356] "Generic (PLEG): container finished" podID="d444a611-792d-4ff0-9cca-cda0856d9577" containerID="bdcba7a08a3978394863e03853be406951624e8a0a71d079a6d5b97824011e0f" exitCode=1
Jul 07 14:56:46.347474 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.347312    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerDied","Data":"bdcba7a08a3978394863e03853be406951624e8a0a71d079a6d5b97824011e0f"}
Jul 07 14:56:46.347474 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.347376    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"e9087f9b976eadf7679679805907aa073ff86191984d300e40a9106ef0ba007d"}
Jul 07 14:56:46.347474 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.347427    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"667e7be41c75ada65a9f85b7280a45d35f315506c8ef6e01ced6f5c2715df675"}
Jul 07 14:56:46.347474 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.347457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"4c149efd29c213d88e990c06fa15416c7058bca4575f8622bdd5166c9cfa2717"}
Jul 07 14:56:46.347662 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.347485    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"11e205aeb4a854f697831645bdc975a3e745a7d4609c204cafba649741028c62"}
Jul 07 14:56:46.348584 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.348562    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d9rmr" event={"ID":"cb5585b7-b224-4b7b-b61b-ccdd7c3165b9","Type":"ContainerStarted","Data":"649d9388358430d2157efd2fb0f962202727d932b3a0a6bceb3843ce0c8f77af"}
Jul 07 14:56:46.366737 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:46.366687    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d9rmr" podStartSLOduration=2.076834803 podStartE2EDuration="22.366670702s" totalImagesPullingTime="20.289835899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:46.36642267 +0000 UTC m=+23.714905301" watchObservedRunningTime="2026-07-07 14:56:46.366670702 +0000 UTC m=+23.715153332"
Jul 07 14:56:47.264808 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.264594    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:47.265007 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:47.264917    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:47.352419 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.352384    2556 generic.go:356] "Generic (PLEG): container finished" podID="7af05a3fbb064bc6b9824d1d785ba3dd" containerID="2d05438c3edea6620be3651a6fe8bffdd924f862ce7c181ba44f89d5f9a0078e" exitCode=0
Jul 07 14:56:47.353023 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.352475    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" event={"ID":"7af05a3fbb064bc6b9824d1d785ba3dd","Type":"ContainerDied","Data":"2d05438c3edea6620be3651a6fe8bffdd924f862ce7c181ba44f89d5f9a0078e"}
Jul 07 14:56:47.354493 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.354464    2556 generic.go:356] "Generic (PLEG): container finished" podID="c0fc80cb-6c79-482c-b8f3-d6856ea7d72c" containerID="a13fa74f0bc097342a67558fb7b7fbcc1b0765d24731912164a7de8fd6cea951" exitCode=0
Jul 07 14:56:47.354590 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.354551    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerDied","Data":"a13fa74f0bc097342a67558fb7b7fbcc1b0765d24731912164a7de8fd6cea951"}
Jul 07 14:56:47.356551 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.356526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9q9q5" event={"ID":"91074283-7c9c-4a20-9196-2d55a7bc6ae1","Type":"ContainerStarted","Data":"411d705485a2f93585e9613af6fc1a5ac61c7451ddf6ebf7222ab3e58d954be3"}
Jul 07 14:56:47.360264 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.360230    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd" event={"ID":"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1","Type":"ContainerStarted","Data":"a198fa71ef04526f19c29f7de75793970ba2149a5fcfcb69fc9eac43140b0717"}
Jul 07 14:56:47.361723 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.361689    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x4xx9" event={"ID":"0b19a2c4-299a-4d9e-a470-50b1d292899f","Type":"ContainerStarted","Data":"619559bf8f11d26b52f8c4392d802b079d6d2fa94d343ab7cc329d8573d36ef5"}
Jul 07 14:56:47.363021 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.363002    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bpb5c" event={"ID":"c835e94b-6053-4750-b83d-354e135e1e8f","Type":"ContainerStarted","Data":"fc79c10b11e039c3b2cf5bcd4e1d6f8156d98b54462c787aa33f771b5023270b"}
Jul 07 14:56:47.417055 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.417004    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9q9q5" podStartSLOduration=3.150989782 podStartE2EDuration="23.416986998s" totalImagesPullingTime="20.265997216s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:47.416760725 +0000 UTC m=+24.765243366" watchObservedRunningTime="2026-07-07 14:56:47.416986998 +0000 UTC m=+24.765469628"
Jul 07 14:56:47.452142 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.452096    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bpb5c" podStartSLOduration=4.347067548 podStartE2EDuration="24.452081325s" totalImagesPullingTime="20.105013777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:47.451904682 +0000 UTC m=+24.800387332" watchObservedRunningTime="2026-07-07 14:56:47.452081325 +0000 UTC m=+24.800563955"
Jul 07 14:56:47.452309 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:47.452180    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-x4xx9" podStartSLOduration=8.704247913 podStartE2EDuration="24.452174119s" totalImagesPullingTime="15.747926206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:47.435119989 +0000 UTC m=+24.783602652" watchObservedRunningTime="2026-07-07 14:56:47.452174119 +0000 UTC m=+24.800656769"
Jul 07 14:56:48.203565 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.203352    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 14:56:48.264742 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.264703    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:48.264742 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.264731    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:48.264967 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:48.264862    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:48.265004 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:48.264978    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:48.369537 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.369443    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd" event={"ID":"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1","Type":"ContainerStarted","Data":"a5fa736c6ba14a6ee5d80a320577fb301a950186721b7cab7509af94462d82ae"}
Jul 07 14:56:48.373545 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.373521    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 14:56:48.373965 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.373939    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"f765c037c1b19552cd43c4767915657ac4a173202b38197d548b019d3ad6a139"}
Jul 07 14:56:48.375729 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.375703    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" event={"ID":"7af05a3fbb064bc6b9824d1d785ba3dd","Type":"ContainerStarted","Data":"212de86041d1ca106ff0d49bfacb7f86e15a34a614064d2d3a761b1f02c5dbb7"}
Jul 07 14:56:48.472118 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.472083    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:48.472845 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.472824    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:48.496247 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:48.496205    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" podStartSLOduration=24.496193093 podStartE2EDuration="24.496193093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:48.393749734 +0000 UTC m=+25.742232365" watchObservedRunningTime="2026-07-07 14:56:48.496193093 +0000 UTC m=+25.844675722"
Jul 07 14:56:49.159540 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:49.159432    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:56:48.203369927Z","UUID":"9a0921e7-bebc-4bd5-9412-b63a2efa1208","Handler":null,"Name":"","Endpoint":""}
Jul 07 14:56:49.161486 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:49.161467    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 14:56:49.161615 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:49.161507    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 14:56:49.268658 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:49.268631    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:49.268860 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:49.268761    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:49.378226 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:49.378192    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:49.378735 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:49.378707    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bpb5c"
Jul 07 14:56:50.264230 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.264004    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:50.264363 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.264054    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:50.264363 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:50.264249    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:50.264363 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:50.264324    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:50.383478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.383446    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 14:56:50.384077 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.383861    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"2e0d53f64936a5e217cb10596c20b185cd7ba534176d38bfdd84771bf901db0b"}
Jul 07 14:56:50.384137 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.384084    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:50.384185 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.384142    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:50.384298 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.384279    2556 scope.go:117] "RemoveContainer" containerID="bdcba7a08a3978394863e03853be406951624e8a0a71d079a6d5b97824011e0f"
Jul 07 14:56:50.386175 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.386146    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd" event={"ID":"cb1c7fbf-4dc9-4066-b7ec-c2bea75935a1","Type":"ContainerStarted","Data":"2682b7e3a2dd7ba1f2505ee32e11408b0a5ce098d7b833fed8887c8c327effc0"}
Jul 07 14:56:50.399683 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.399663    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:50.443032 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:50.442986    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlcdd" podStartSLOduration=3.331753038 podStartE2EDuration="26.442969382s" totalImagesPullingTime="23.111216344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:50.44244972 +0000 UTC m=+27.790932382" watchObservedRunningTime="2026-07-07 14:56:50.442969382 +0000 UTC m=+27.791452015"
Jul 07 14:56:51.268778 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.268588    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:51.268778 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:51.268722    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:51.392082 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.392057    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 14:56:51.392661 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.392532    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" event={"ID":"d444a611-792d-4ff0-9cca-cda0856d9577","Type":"ContainerStarted","Data":"c4f103ff0bcbfdd9e75b3f8e490551e16a04185930d5842669848031939e85f8"}
Jul 07 14:56:51.392889 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.392863    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:51.408556 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.408531    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:56:51.425269 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.425216    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw" podStartSLOduration=8.369979353 podStartE2EDuration="28.425201252s" totalImagesPullingTime="20.055221899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:51.424639649 +0000 UTC m=+28.773122298" watchObservedRunningTime="2026-07-07 14:56:51.425201252 +0000 UTC m=+28.773683882"
Jul 07 14:56:51.940918 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.940697    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-756dp"]
Jul 07 14:56:51.941095 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.941014    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:51.941184 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:51.941142    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:51.945183 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.945120    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f7mm6"]
Jul 07 14:56:51.945346 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.945280    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:51.945490 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:51.945445    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:51.945995 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.945962    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxd9k"]
Jul 07 14:56:51.946115 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:51.946060    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:51.946194 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:51.946160    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:53.267993 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:53.267962    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:53.268422 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:53.268074    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:53.268422 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:53.268134    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:53.268422 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:53.268252    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:54.263948 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:54.263911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:54.264089 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:54.264026    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:55.264497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:55.264459    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:55.264497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:55.264473    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:55.265171 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.264568    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:55.265171 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.264687    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:55.406458 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:55.406418    2556 generic.go:356] "Generic (PLEG): container finished" podID="c0fc80cb-6c79-482c-b8f3-d6856ea7d72c" containerID="573dab84600cb2a402b8442adf9ffb3bc0fcd2e45a68e3561d77b080e0e54ebe" exitCode=0
Jul 07 14:56:55.406656 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:55.406475    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerDied","Data":"573dab84600cb2a402b8442adf9ffb3bc0fcd2e45a68e3561d77b080e0e54ebe"}
Jul 07 14:56:55.953632 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:55.953594    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:55.953837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:55.953648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:55.953837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:55.953700    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:55.953837 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953734    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:55.953837 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953779    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:55.953837 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953816    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:27.953781103 +0000 UTC m=+65.302263718 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:55.953837 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953835    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret podName:dec33f92-47f6-4040-8a3b-4658fdbf72ca nodeName:}" failed. No retries permitted until 2026-07-07 14:57:27.95382554 +0000 UTC m=+65.302308148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret") pod "global-pull-secret-syncer-f7mm6" (UID: "dec33f92-47f6-4040-8a3b-4658fdbf72ca") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:55.954116 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953848    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:55.954116 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953873    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:55.954116 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953886    2556 projected.go:196] Error preparing data for projected volume kube-api-access-wv4c5 for pod openshift-network-diagnostics/network-check-target-756dp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:55.954116 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:55.953940    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5 podName:70153084-d026-4d8c-a91c-dff91ff1f515 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:27.953923769 +0000 UTC m=+65.302406390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wv4c5" (UniqueName: "kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5") pod "network-check-target-756dp" (UID: "70153084-d026-4d8c-a91c-dff91ff1f515") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:56.264319 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:56.264238    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:56.264451 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:56.264340    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7mm6" podUID="dec33f92-47f6-4040-8a3b-4658fdbf72ca"
Jul 07 14:56:57.267276 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:57.267099    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:57.267713 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:57.267151    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:57.267713 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:57.267373    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxd9k" podUID="b6fd483f-453e-4289-82fb-02ad4d241017"
Jul 07 14:56:57.267713 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:57.267396    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-756dp" podUID="70153084-d026-4d8c-a91c-dff91ff1f515"
Jul 07 14:56:57.412729 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:57.412695    2556 generic.go:356] "Generic (PLEG): container finished" podID="c0fc80cb-6c79-482c-b8f3-d6856ea7d72c" containerID="89b2df1491521b0b979927fa048e174bdc4ae9db5cfd5812e21b67267093b60e" exitCode=0
Jul 07 14:56:57.412729 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:57.412731    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerDied","Data":"89b2df1491521b0b979927fa048e174bdc4ae9db5cfd5812e21b67267093b60e"}
Jul 07 14:56:57.969355 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:57.969328    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeReady"
Jul 07 14:56:57.969507 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:57.969438    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 14:56:58.004444 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.004414    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78c556786f-mfrd7"]
Jul 07 14:56:58.025728 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.025702    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78c556786f-mfrd7"]
Jul 07 14:56:58.025885 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.025740    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dht8q"]
Jul 07 14:56:58.025885 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.025840    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.028312 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.028291    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 14:56:58.028429 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.028384    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 14:56:58.028429 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.028408    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mrpzk\""
Jul 07 14:56:58.028879 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.028864    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 14:56:58.035126 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.035107    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 14:56:58.041974 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.041952    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xjl76"]
Jul 07 14:56:58.042070 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.042046    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.045083 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.045058    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 14:56:58.045196 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.045104    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bbd6b\""
Jul 07 14:56:58.045476 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.045458    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 14:56:58.050209 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.050148    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 14:56:58.056160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.056138    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dht8q"]
Jul 07 14:56:58.056160 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.056164    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xjl76"]
Jul 07 14:56:58.056281 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.056259    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.058483 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.058462    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 14:56:58.058584 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.058562    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 14:56:58.058687 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.058671    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pkm7x\""
Jul 07 14:56:58.167239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167196    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-installation-pull-secrets\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.167239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167239    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/012cad90-75d4-490c-82fe-fba736164700-ca-trust-extracted\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.167479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167315    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-bound-sa-token\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.167479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167361    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0c5e587-0e58-4aa8-b704-350d87c04978-cert\") pod \"ingress-canary-dht8q\" (UID: \"c0c5e587-0e58-4aa8-b704-350d87c04978\") " pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.167479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167431    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-image-registry-private-configuration\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.167479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167455    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrdsx\" (UniqueName: \"kubernetes.io/projected/0718570a-e9fe-460b-a0c9-b895125c2d44-kube-api-access-jrdsx\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.167479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167471    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-registry-certificates\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.167769 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167490    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.167769 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167548    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0718570a-e9fe-460b-a0c9-b895125c2d44-config-volume\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.167769 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167581    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.167769 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167602    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0718570a-e9fe-460b-a0c9-b895125c2d44-tmp-dir\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.167769 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167619    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2vld\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-kube-api-access-d2vld\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.167769 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167634    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4226\" (UniqueName: \"kubernetes.io/projected/c0c5e587-0e58-4aa8-b704-350d87c04978-kube-api-access-w4226\") pod \"ingress-canary-dht8q\" (UID: \"c0c5e587-0e58-4aa8-b704-350d87c04978\") " pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.167769 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.167650    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-trusted-ca\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.191290 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.191240    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:56:58.264670 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.264626    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:56:58.267412 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.267390    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:56:58.268691 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268658    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-image-registry-private-configuration\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.268837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268691    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrdsx\" (UniqueName: \"kubernetes.io/projected/0718570a-e9fe-460b-a0c9-b895125c2d44-kube-api-access-jrdsx\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.268837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268715    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-registry-certificates\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.268837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268734    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.268837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268769    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0718570a-e9fe-460b-a0c9-b895125c2d44-config-volume\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.268837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268812    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.268837 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268836    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0718570a-e9fe-460b-a0c9-b895125c2d44-tmp-dir\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268858    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2vld\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-kube-api-access-d2vld\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4226\" (UniqueName: \"kubernetes.io/projected/c0c5e587-0e58-4aa8-b704-350d87c04978-kube-api-access-w4226\") pod \"ingress-canary-dht8q\" (UID: \"c0c5e587-0e58-4aa8-b704-350d87c04978\") " pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268908    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-trusted-ca\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268945    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-installation-pull-secrets\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.268986    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/012cad90-75d4-490c-82fe-fba736164700-ca-trust-extracted\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.268999    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.269016    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c556786f-mfrd7: secret "image-registry-tls" not found
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.269027    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-bound-sa-token\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.269048    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0c5e587-0e58-4aa8-b704-350d87c04978-cert\") pod \"ingress-canary-dht8q\" (UID: \"c0c5e587-0e58-4aa8-b704-350d87c04978\") " pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.269131 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.269074    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls podName:012cad90-75d4-490c-82fe-fba736164700 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:58.769053202 +0000 UTC m=+36.117535820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls") pod "image-registry-78c556786f-mfrd7" (UID: "012cad90-75d4-490c-82fe-fba736164700") : secret "image-registry-tls" not found
Jul 07 14:56:58.269573 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.269238    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:56:58.269573 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.269242    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0718570a-e9fe-460b-a0c9-b895125c2d44-tmp-dir\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.269573 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.269291    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls podName:0718570a-e9fe-460b-a0c9-b895125c2d44 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:58.769272548 +0000 UTC m=+36.117755159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls") pod "dns-default-xjl76" (UID: "0718570a-e9fe-460b-a0c9-b895125c2d44") : secret "dns-default-metrics-tls" not found
Jul 07 14:56:58.269734 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.269624    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0718570a-e9fe-460b-a0c9-b895125c2d44-config-volume\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.269932 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.269887    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/012cad90-75d4-490c-82fe-fba736164700-ca-trust-extracted\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.270060 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.270016    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-registry-certificates\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.270302 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.270278    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-trusted-ca\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.273989 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.273961    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-installation-pull-secrets\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.273989 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.273981    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0c5e587-0e58-4aa8-b704-350d87c04978-cert\") pod \"ingress-canary-dht8q\" (UID: \"c0c5e587-0e58-4aa8-b704-350d87c04978\") " pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.274126 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.274055    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-image-registry-private-configuration\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.278986 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.278936    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2vld\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-kube-api-access-d2vld\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.279440 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.279407    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrdsx\" (UniqueName: \"kubernetes.io/projected/0718570a-e9fe-460b-a0c9-b895125c2d44-kube-api-access-jrdsx\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.279440 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.279433    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-bound-sa-token\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.280078 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.280056    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4226\" (UniqueName: \"kubernetes.io/projected/c0c5e587-0e58-4aa8-b704-350d87c04978-kube-api-access-w4226\") pod \"ingress-canary-dht8q\" (UID: \"c0c5e587-0e58-4aa8-b704-350d87c04978\") " pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.290470 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.290435    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:56:58.353440 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.353370    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dht8q"
Jul 07 14:56:58.537108 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.537077    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dht8q"]
Jul 07 14:56:58.579259 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:56:58.579229    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c5e587_0e58_4aa8_b704_350d87c04978.slice/crio-6862a435163cb151b32dd6fb4941b0e61c71e291aa67be4c1bda47c0fe22da47 WatchSource:0}: Error finding container 6862a435163cb151b32dd6fb4941b0e61c71e291aa67be4c1bda47c0fe22da47: Status 404 returned error can't find the container with id 6862a435163cb151b32dd6fb4941b0e61c71e291aa67be4c1bda47c0fe22da47
Jul 07 14:56:58.773510 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.773360    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:58.773600 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:58.773538    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:58.773600 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.773492    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:56:58.773600 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.773594    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c556786f-mfrd7: secret "image-registry-tls" not found
Jul 07 14:56:58.773692 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.773642    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:56:58.773692 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.773648    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls podName:012cad90-75d4-490c-82fe-fba736164700 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:59.773630523 +0000 UTC m=+37.122113131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls") pod "image-registry-78c556786f-mfrd7" (UID: "012cad90-75d4-490c-82fe-fba736164700") : secret "image-registry-tls" not found
Jul 07 14:56:58.773758 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:58.773693    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls podName:0718570a-e9fe-460b-a0c9-b895125c2d44 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:59.773681179 +0000 UTC m=+37.122163788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls") pod "dns-default-xjl76" (UID: "0718570a-e9fe-460b-a0c9-b895125c2d44") : secret "dns-default-metrics-tls" not found
Jul 07 14:56:59.074979 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.074943    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:56:59.175213 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.175178    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:56:59.268463 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.268427    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:56:59.269254 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.268433    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:56:59.271081 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.271050    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:56:59.271400 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.271384    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:56:59.272078 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.272047    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:56:59.272078 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.272052    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8fcz7\""
Jul 07 14:56:59.272237 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.272079    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jrv9k\""
Jul 07 14:56:59.287479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.287422    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:56:59.378787 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.378704    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:56:59.420615 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.420581    2556 generic.go:356] "Generic (PLEG): container finished" podID="c0fc80cb-6c79-482c-b8f3-d6856ea7d72c" containerID="7731e8695e6f5cb4f786f5664c160d67e7716493d7fd882a8832a7f39212f93b" exitCode=0
Jul 07 14:56:59.420828 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.420673    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerDied","Data":"7731e8695e6f5cb4f786f5664c160d67e7716493d7fd882a8832a7f39212f93b"}
Jul 07 14:56:59.422118 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.422091    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dht8q" event={"ID":"c0c5e587-0e58-4aa8-b704-350d87c04978","Type":"ContainerStarted","Data":"6862a435163cb151b32dd6fb4941b0e61c71e291aa67be4c1bda47c0fe22da47"}
Jul 07 14:56:59.458348 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.458312    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:56:59.780327 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.780296    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:56:59.780547 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:56:59.780364    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:56:59.780547 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:59.780447    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:56:59.780547 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:59.780457    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c556786f-mfrd7: secret "image-registry-tls" not found
Jul 07 14:56:59.780547 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:59.780445    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:56:59.780674 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:59.780517    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls podName:012cad90-75d4-490c-82fe-fba736164700 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:01.780503792 +0000 UTC m=+39.128986399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls") pod "image-registry-78c556786f-mfrd7" (UID: "012cad90-75d4-490c-82fe-fba736164700") : secret "image-registry-tls" not found
Jul 07 14:56:59.780674 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:56:59.780587    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls podName:0718570a-e9fe-460b-a0c9-b895125c2d44 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:01.780574559 +0000 UTC m=+39.129057167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls") pod "dns-default-xjl76" (UID: "0718570a-e9fe-460b-a0c9-b895125c2d44") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:00.058060 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:00.057982    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:00.157974 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:00.157939    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:00.444057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:00.444026    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:00.543449 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:00.543401    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:01.797548 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:01.797457    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:57:01.797548 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:01.797508    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:57:01.798028 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:01.797628    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:01.798028 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:01.797683    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls podName:0718570a-e9fe-460b-a0c9-b895125c2d44 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:05.79767003 +0000 UTC m=+43.146152639 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls") pod "dns-default-xjl76" (UID: "0718570a-e9fe-460b-a0c9-b895125c2d44") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:01.798028 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:01.797631    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:01.798028 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:01.797745    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c556786f-mfrd7: secret "image-registry-tls" not found
Jul 07 14:57:01.798028 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:01.797843    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls podName:012cad90-75d4-490c-82fe-fba736164700 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:05.797822349 +0000 UTC m=+43.146304979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls") pod "image-registry-78c556786f-mfrd7" (UID: "012cad90-75d4-490c-82fe-fba736164700") : secret "image-registry-tls" not found
Jul 07 14:57:02.431211 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:02.431175    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dht8q" event={"ID":"c0c5e587-0e58-4aa8-b704-350d87c04978","Type":"ContainerStarted","Data":"f0709150ba397b9b773590fb96ee49941edffd246903b7725312377177d23dcd"}
Jul 07 14:57:02.449327 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:02.449274    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dht8q" podStartSLOduration=1.128698621 podStartE2EDuration="4.449255906s" totalImagesPullingTime="3.320557285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:02.448517599 +0000 UTC m=+39.797000250" watchObservedRunningTime="2026-07-07 14:57:02.449255906 +0000 UTC m=+39.797738535"
Jul 07 14:57:03.420409 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:03.420372    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:03.485696 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:03.484245    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:05.441318 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:05.441292    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerStarted","Data":"4795e373d163c1226b8d2f00ebbb0f82b883a8310229f7e80b334b2b281827dc"}
Jul 07 14:57:05.828710 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:05.828668    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:57:05.828918 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:05.828727    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:57:05.828918 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:05.828840    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:05.828918 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:05.828859    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c556786f-mfrd7: secret "image-registry-tls" not found
Jul 07 14:57:05.828918 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:05.828893    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:05.828918 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:05.828911    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls podName:012cad90-75d4-490c-82fe-fba736164700 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:13.828896307 +0000 UTC m=+51.177378920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls") pod "image-registry-78c556786f-mfrd7" (UID: "012cad90-75d4-490c-82fe-fba736164700") : secret "image-registry-tls" not found
Jul 07 14:57:05.829090 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:05.828944    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls podName:0718570a-e9fe-460b-a0c9-b895125c2d44 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:13.828931498 +0000 UTC m=+51.177414107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls") pod "dns-default-xjl76" (UID: "0718570a-e9fe-460b-a0c9-b895125c2d44") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:05.853293 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:05.853254    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:06.446011 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:06.445969    2556 generic.go:356] "Generic (PLEG): container finished" podID="c0fc80cb-6c79-482c-b8f3-d6856ea7d72c" containerID="4795e373d163c1226b8d2f00ebbb0f82b883a8310229f7e80b334b2b281827dc" exitCode=0
Jul 07 14:57:06.446382 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:06.446016    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerDied","Data":"4795e373d163c1226b8d2f00ebbb0f82b883a8310229f7e80b334b2b281827dc"}
Jul 07 14:57:07.454982 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:07.454943    2556 generic.go:356] "Generic (PLEG): container finished" podID="c0fc80cb-6c79-482c-b8f3-d6856ea7d72c" containerID="72efe8c4a28cfb7729d5e6d9949d8bc46e9426ce0f503e39a26d58f056a802be" exitCode=0
Jul 07 14:57:07.455343 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:07.455000    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerDied","Data":"72efe8c4a28cfb7729d5e6d9949d8bc46e9426ce0f503e39a26d58f056a802be"}
Jul 07 14:57:08.461699 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:08.461669    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" event={"ID":"c0fc80cb-6c79-482c-b8f3-d6856ea7d72c","Type":"ContainerStarted","Data":"9a67a9b1366e3c44d4f078ef8d934e1f0635f94e1bcc985cf193145de2132e43"}
Jul 07 14:57:08.488625 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:08.488561    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n6gbg" podStartSLOduration=10.217776588 podStartE2EDuration="45.488545165s" totalImagesPullingTime="35.270768577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:08.48702638 +0000 UTC m=+45.835509029" watchObservedRunningTime="2026-07-07 14:57:08.488545165 +0000 UTC m=+45.837027798"
Jul 07 14:57:13.886447 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:13.886375    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:57:13.886447 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:13.886424    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:57:13.886814 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:13.886531    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:13.886814 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:13.886539    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:13.886814 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:13.886562    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c556786f-mfrd7: secret "image-registry-tls" not found
Jul 07 14:57:13.886814 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:13.886594    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls podName:0718570a-e9fe-460b-a0c9-b895125c2d44 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:29.886578158 +0000 UTC m=+67.235060784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls") pod "dns-default-xjl76" (UID: "0718570a-e9fe-460b-a0c9-b895125c2d44") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:13.886814 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:13.886610    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls podName:012cad90-75d4-490c-82fe-fba736164700 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:29.88660333 +0000 UTC m=+67.235085937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls") pod "image-registry-78c556786f-mfrd7" (UID: "012cad90-75d4-490c-82fe-fba736164700") : secret "image-registry-tls" not found
Jul 07 14:57:18.125022 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.124975    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"]
Jul 07 14:57:18.128071 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.128043    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.130362 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.130326    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 14:57:18.130500 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.130380    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 07 14:57:18.132657 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.132616    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ggbvw\""
Jul 07 14:57:18.132657 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.132640    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 14:57:18.132657 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.132638    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 07 14:57:18.136751 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.136723    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"]
Jul 07 14:57:18.216857 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.216829    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.216857 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.216858    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz7zs\" (UniqueName: \"kubernetes.io/projected/0227a3b2-bc6c-4274-91f5-5dee553832d3-kube-api-access-wz7zs\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.217049 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.216883    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0227a3b2-bc6c-4274-91f5-5dee553832d3-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.220230 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.220204    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-n5ptf"]
Jul 07 14:57:18.222177 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.222161    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.223800 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.223773    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5f8f66bc9c-fhp6v"]
Jul 07 14:57:18.224497 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.224480    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 07 14:57:18.224602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.224581    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4vrlg\""
Jul 07 14:57:18.224727 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.224697    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 07 14:57:18.224891 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.224854    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 14:57:18.224977 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.224958    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 14:57:18.225825 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.225806    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.228521 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.228499    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 14:57:18.228917 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.228898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bl5fp\""
Jul 07 14:57:18.229020 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.228997    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 07 14:57:18.229020 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.229016    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 07 14:57:18.229162 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.229028    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 14:57:18.229227 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.229212    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 07 14:57:18.229648 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.229453    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 07 14:57:18.231484 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.231460    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 07 14:57:18.232840 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.232812    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-n5ptf"]
Jul 07 14:57:18.239930 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.239904    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f8f66bc9c-fhp6v"]
Jul 07 14:57:18.317836 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.317802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2992c03-9dce-432d-a63e-99a5c3723d06-tmp\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.317960 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.317862    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz7zs\" (UniqueName: \"kubernetes.io/projected/0227a3b2-bc6c-4274-91f5-5dee553832d3-kube-api-access-wz7zs\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.317960 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.317895    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-stats-auth\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.317960 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.317914    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.317960 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.317931    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhfw\" (UniqueName: \"kubernetes.io/projected/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-kube-api-access-fdhfw\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.317960 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.317952    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2992c03-9dce-432d-a63e-99a5c3723d06-serving-cert\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.317979    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-default-certificate\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318012    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318051    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318101    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c2992c03-9dce-432d-a63e-99a5c3723d06-snapshots\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.318122    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318137    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0227a3b2-bc6c-4274-91f5-5dee553832d3-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318165    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjb4x\" (UniqueName: \"kubernetes.io/projected/c2992c03-9dce-432d-a63e-99a5c3723d06-kube-api-access-hjb4x\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.318195 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.318179    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls podName:0227a3b2-bc6c-4274-91f5-5dee553832d3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:18.818164107 +0000 UTC m=+56.166646715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-pvr7j" (UID: "0227a3b2-bc6c-4274-91f5-5dee553832d3") : secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:18.318478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318237    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2992c03-9dce-432d-a63e-99a5c3723d06-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.318478 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318263    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2992c03-9dce-432d-a63e-99a5c3723d06-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.318718 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.318702    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0227a3b2-bc6c-4274-91f5-5dee553832d3-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.328724 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.328699    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz7zs\" (UniqueName: \"kubernetes.io/projected/0227a3b2-bc6c-4274-91f5-5dee553832d3-kube-api-access-wz7zs\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.418878 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.418856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-stats-auth\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.418970 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.418882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.418970 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.418899    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhfw\" (UniqueName: \"kubernetes.io/projected/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-kube-api-access-fdhfw\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.418970 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.418920    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2992c03-9dce-432d-a63e-99a5c3723d06-serving-cert\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.419106 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.418979    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-default-certificate\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.419106 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.418997    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:57:18.419106 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419017    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.419106 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.419038    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:18.919026274 +0000 UTC m=+56.267508883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : secret "router-metrics-certs-default" not found
Jul 07 14:57:18.419106 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.419105    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:18.919091228 +0000 UTC m=+56.267573838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : configmap references non-existent config key: service-ca.crt
Jul 07 14:57:18.419344 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419143    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c2992c03-9dce-432d-a63e-99a5c3723d06-snapshots\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.419344 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjb4x\" (UniqueName: \"kubernetes.io/projected/c2992c03-9dce-432d-a63e-99a5c3723d06-kube-api-access-hjb4x\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.419344 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2992c03-9dce-432d-a63e-99a5c3723d06-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.419344 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419241    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2992c03-9dce-432d-a63e-99a5c3723d06-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.419344 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419281    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2992c03-9dce-432d-a63e-99a5c3723d06-tmp\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.419726 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419702    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2992c03-9dce-432d-a63e-99a5c3723d06-tmp\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.419915 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419758    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c2992c03-9dce-432d-a63e-99a5c3723d06-snapshots\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.420024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.419952    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2992c03-9dce-432d-a63e-99a5c3723d06-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.420276 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.420256    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2992c03-9dce-432d-a63e-99a5c3723d06-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.421303 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.421266    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-default-certificate\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.421640 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.421625    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2992c03-9dce-432d-a63e-99a5c3723d06-serving-cert\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.421892 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.421870    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-stats-auth\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.429184 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.429159    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhfw\" (UniqueName: \"kubernetes.io/projected/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-kube-api-access-fdhfw\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.429438 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.429422    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjb4x\" (UniqueName: \"kubernetes.io/projected/c2992c03-9dce-432d-a63e-99a5c3723d06-kube-api-access-hjb4x\") pod \"insights-operator-5c847dcfc4-n5ptf\" (UID: \"c2992c03-9dce-432d-a63e-99a5c3723d06\") " pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.533677 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.533657    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf"
Jul 07 14:57:18.648471 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.648444    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-n5ptf"]
Jul 07 14:57:18.652301 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:18.652278    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2992c03_9dce_432d_a63e_99a5c3723d06.slice/crio-a0904fd41682ef320fc490f88f9377944066f143ee379297dda4c33567482166 WatchSource:0}: Error finding container a0904fd41682ef320fc490f88f9377944066f143ee379297dda4c33567482166: Status 404 returned error can't find the container with id a0904fd41682ef320fc490f88f9377944066f143ee379297dda4c33567482166
Jul 07 14:57:18.821992 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.821928    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:18.822146 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.822028    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:18.822146 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.822076    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls podName:0227a3b2-bc6c-4274-91f5-5dee553832d3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:19.822063977 +0000 UTC m=+57.170546585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-pvr7j" (UID: "0227a3b2-bc6c-4274-91f5-5dee553832d3") : secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:18.922668 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.922632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.922776 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:18.922761    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:18.922854 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.922839    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:19.922773233 +0000 UTC m=+57.271255846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : configmap references non-existent config key: service-ca.crt
Jul 07 14:57:18.922909 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.922873    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:57:18.922943 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:18.922922    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:19.922907421 +0000 UTC m=+57.271390045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : secret "router-metrics-certs-default" not found
Jul 07 14:57:19.132009 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.131954    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"]
Jul 07 14:57:19.134379 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.134364    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:19.138461 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.138436    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 14:57:19.138559 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.138479    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 14:57:19.138559 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.138523    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gbm9l\""
Jul 07 14:57:19.138951 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.138932    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:19.150407 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.147476    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"]
Jul 07 14:57:19.224959 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.224938    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:19.225060 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.224987    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xgfz\" (UniqueName: \"kubernetes.io/projected/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-kube-api-access-5xgfz\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:19.234560 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.234537    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"]
Jul 07 14:57:19.236287 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.236273    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"
Jul 07 14:57:19.240904 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.240882    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7f7k8\""
Jul 07 14:57:19.241018 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.240886    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 07 14:57:19.241089 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.241039    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:19.249777 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.249753    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"]
Jul 07 14:57:19.325955 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.325927    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4kmf\" (UniqueName: \"kubernetes.io/projected/02ccf314-fc77-4a3e-b795-883a2a897ba5-kube-api-access-c4kmf\") pod \"volume-data-source-validator-5d456fdbcd-8p572\" (UID: \"02ccf314-fc77-4a3e-b795-883a2a897ba5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"
Jul 07 14:57:19.326047 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.325962    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xgfz\" (UniqueName: \"kubernetes.io/projected/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-kube-api-access-5xgfz\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:19.326092 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.326049    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:19.326170 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.326156    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:57:19.326225 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.326213    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls podName:6b491dac-474d-4cc9-9ba3-c9eca1a04b5e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:19.826198382 +0000 UTC m=+57.174680993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zsr8" (UID: "6b491dac-474d-4cc9-9ba3-c9eca1a04b5e") : secret "samples-operator-tls" not found
Jul 07 14:57:19.337349 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.337327    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xgfz\" (UniqueName: \"kubernetes.io/projected/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-kube-api-access-5xgfz\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:19.426477 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.426454    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4kmf\" (UniqueName: \"kubernetes.io/projected/02ccf314-fc77-4a3e-b795-883a2a897ba5-kube-api-access-c4kmf\") pod \"volume-data-source-validator-5d456fdbcd-8p572\" (UID: \"02ccf314-fc77-4a3e-b795-883a2a897ba5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"
Jul 07 14:57:19.435368 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.435345    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4kmf\" (UniqueName: \"kubernetes.io/projected/02ccf314-fc77-4a3e-b795-883a2a897ba5-kube-api-access-c4kmf\") pod \"volume-data-source-validator-5d456fdbcd-8p572\" (UID: \"02ccf314-fc77-4a3e-b795-883a2a897ba5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"
Jul 07 14:57:19.487018 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.486992    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf" event={"ID":"c2992c03-9dce-432d-a63e-99a5c3723d06","Type":"ContainerStarted","Data":"a0904fd41682ef320fc490f88f9377944066f143ee379297dda4c33567482166"}
Jul 07 14:57:19.543843 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.543823    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"
Jul 07 14:57:19.667119 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.667094    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572"]
Jul 07 14:57:19.669755 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:19.669731    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ccf314_fc77_4a3e_b795_883a2a897ba5.slice/crio-e296806d4f61566bc47f7ff9acbc97285be7dc8dea39788d494b1d4bf76a6d2b WatchSource:0}: Error finding container e296806d4f61566bc47f7ff9acbc97285be7dc8dea39788d494b1d4bf76a6d2b: Status 404 returned error can't find the container with id e296806d4f61566bc47f7ff9acbc97285be7dc8dea39788d494b1d4bf76a6d2b
Jul 07 14:57:19.829933 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.829871    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:19.830040 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.829981    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:19.830040 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.830014    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:57:19.830127 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.830065    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls podName:6b491dac-474d-4cc9-9ba3-c9eca1a04b5e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:20.830051105 +0000 UTC m=+58.178533713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zsr8" (UID: "6b491dac-474d-4cc9-9ba3-c9eca1a04b5e") : secret "samples-operator-tls" not found
Jul 07 14:57:19.830127 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.830069    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:19.830127 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.830106    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls podName:0227a3b2-bc6c-4274-91f5-5dee553832d3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:21.830094928 +0000 UTC m=+59.178577536 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-pvr7j" (UID: "0227a3b2-bc6c-4274-91f5-5dee553832d3") : secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:19.930621 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.930598    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:19.930752 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:19.930645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:19.930752 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.930731    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:57:19.930892 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.930810    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:21.930773554 +0000 UTC m=+59.279256166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : secret "router-metrics-certs-default" not found
Jul 07 14:57:19.930892 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:19.930830    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:21.930820059 +0000 UTC m=+59.279302671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : configmap references non-existent config key: service-ca.crt
Jul 07 14:57:20.084268 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.084206    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-k8r9l"]
Jul 07 14:57:20.086999 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.086979    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.089517 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.089494    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 07 14:57:20.089616 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.089534    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kc8xz\""
Jul 07 14:57:20.089616 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.089605    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 07 14:57:20.089945 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.089922    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 07 14:57:20.090024 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.089928    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:20.096490 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.096471    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 07 14:57:20.102089 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.102069    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-k8r9l"]
Jul 07 14:57:20.232982 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.232957    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tkwl\" (UniqueName: \"kubernetes.io/projected/77040db7-a4f5-4195-8504-062e41f25c55-kube-api-access-7tkwl\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.233313 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.233022    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77040db7-a4f5-4195-8504-062e41f25c55-trusted-ca\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.233313 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.233104    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77040db7-a4f5-4195-8504-062e41f25c55-config\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.233313 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.233173    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77040db7-a4f5-4195-8504-062e41f25c55-serving-cert\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.334553 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.334502    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tkwl\" (UniqueName: \"kubernetes.io/projected/77040db7-a4f5-4195-8504-062e41f25c55-kube-api-access-7tkwl\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.334553 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.334547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77040db7-a4f5-4195-8504-062e41f25c55-trusted-ca\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.334709 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.334673    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77040db7-a4f5-4195-8504-062e41f25c55-config\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.334764 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.334709    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77040db7-a4f5-4195-8504-062e41f25c55-serving-cert\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.335269 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.335244    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77040db7-a4f5-4195-8504-062e41f25c55-config\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.335359 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.335276    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77040db7-a4f5-4195-8504-062e41f25c55-trusted-ca\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.336980 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.336957    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77040db7-a4f5-4195-8504-062e41f25c55-serving-cert\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.343239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.343218    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tkwl\" (UniqueName: \"kubernetes.io/projected/77040db7-a4f5-4195-8504-062e41f25c55-kube-api-access-7tkwl\") pod \"console-operator-68b757fb67-k8r9l\" (UID: \"77040db7-a4f5-4195-8504-062e41f25c55\") " pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.396972 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.396946    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:20.492413 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.492355    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572" event={"ID":"02ccf314-fc77-4a3e-b795-883a2a897ba5","Type":"ContainerStarted","Data":"e296806d4f61566bc47f7ff9acbc97285be7dc8dea39788d494b1d4bf76a6d2b"}
Jul 07 14:57:20.527552 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.527525    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-k8r9l"]
Jul 07 14:57:20.530495 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:20.530471    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77040db7_a4f5_4195_8504_062e41f25c55.slice/crio-e9a21fdba7059cc1bce2e52f83f52163737bfc75f02f6965e36e011932624ed3 WatchSource:0}: Error finding container e9a21fdba7059cc1bce2e52f83f52163737bfc75f02f6965e36e011932624ed3: Status 404 returned error can't find the container with id e9a21fdba7059cc1bce2e52f83f52163737bfc75f02f6965e36e011932624ed3
Jul 07 14:57:20.839099 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:20.839065    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:20.839272 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:20.839226    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:57:20.839338 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:20.839315    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls podName:6b491dac-474d-4cc9-9ba3-c9eca1a04b5e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:22.839294296 +0000 UTC m=+60.187776907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zsr8" (UID: "6b491dac-474d-4cc9-9ba3-c9eca1a04b5e") : secret "samples-operator-tls" not found
Jul 07 14:57:21.497113 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.497073    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf" event={"ID":"c2992c03-9dce-432d-a63e-99a5c3723d06","Type":"ContainerStarted","Data":"a06bb1cfba65358f47d33ceb739e6a81d816fa12b879a0b9c06decee2e112d5e"}
Jul 07 14:57:21.499267 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.499101    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572" event={"ID":"02ccf314-fc77-4a3e-b795-883a2a897ba5","Type":"ContainerStarted","Data":"885da58dddafd49dd6051ac58d376fe6b6471308eb0aad56d4d49d33d6129c3a"}
Jul 07 14:57:21.500831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.500783    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" event={"ID":"77040db7-a4f5-4195-8504-062e41f25c55","Type":"ContainerStarted","Data":"e9a21fdba7059cc1bce2e52f83f52163737bfc75f02f6965e36e011932624ed3"}
Jul 07 14:57:21.516645 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.516601    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf" podStartSLOduration=0.826977238 podStartE2EDuration="3.51658384s" totalImagesPullingTime="2.689606602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:21.516065427 +0000 UTC m=+58.864548081" watchObservedRunningTime="2026-07-07 14:57:21.51658384 +0000 UTC m=+58.865066473"
Jul 07 14:57:21.535633 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.535441    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8p572" podStartSLOduration=0.863752275 podStartE2EDuration="2.535424824s" totalImagesPullingTime="1.671672549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:21.534938208 +0000 UTC m=+58.883420838" watchObservedRunningTime="2026-07-07 14:57:21.535424824 +0000 UTC m=+58.883907454"
Jul 07 14:57:21.845781 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.845748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:21.845966 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:21.845949    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:21.846033 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:21.846014    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls podName:0227a3b2-bc6c-4274-91f5-5dee553832d3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:25.845996644 +0000 UTC m=+63.194479267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-pvr7j" (UID: "0227a3b2-bc6c-4274-91f5-5dee553832d3") : secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:21.947137 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.947110    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:21.947137 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:21.947134    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:57:21.947313 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:21.947185    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:21.947313 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:21.947192    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:25.947179904 +0000 UTC m=+63.295662512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : secret "router-metrics-certs-default" not found
Jul 07 14:57:21.947399 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:21.947353    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:25.947340319 +0000 UTC m=+63.295822944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : configmap references non-existent config key: service-ca.crt
Jul 07 14:57:22.854997 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:22.854960    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:22.855386 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:22.855113    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:57:22.855386 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:22.855188    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls podName:6b491dac-474d-4cc9-9ba3-c9eca1a04b5e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:26.855167965 +0000 UTC m=+64.203650582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zsr8" (UID: "6b491dac-474d-4cc9-9ba3-c9eca1a04b5e") : secret "samples-operator-tls" not found
Jul 07 14:57:23.052682 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.052649    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6mb4h"]
Jul 07 14:57:23.058002 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.057978    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.062098 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.062070    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-85x7v\""
Jul 07 14:57:23.063029 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.062860    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 14:57:23.063029 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.062889    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 14:57:23.074605 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.074564    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mb4h"]
Jul 07 14:57:23.158745 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.158671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.158745 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.158703    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxdp8\" (UniqueName: \"kubernetes.io/projected/71190e89-69d2-4d28-8825-f3de21d2de28-kube-api-access-rxdp8\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.158950 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.158830    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/71190e89-69d2-4d28-8825-f3de21d2de28-data-volume\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.158950 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.158914    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/71190e89-69d2-4d28-8825-f3de21d2de28-crio-socket\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.159055 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.158949    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.259449 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.259393    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/71190e89-69d2-4d28-8825-f3de21d2de28-crio-socket\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.259449 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.259439    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.259651 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.259525    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:23.259651 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.259531    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/71190e89-69d2-4d28-8825-f3de21d2de28-crio-socket\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.259651 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.259580    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:23.759563235 +0000 UTC m=+61.108045860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:23.259651 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.259645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.259885 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.259682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxdp8\" (UniqueName: \"kubernetes.io/projected/71190e89-69d2-4d28-8825-f3de21d2de28-kube-api-access-rxdp8\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.259885 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.259759    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/71190e89-69d2-4d28-8825-f3de21d2de28-data-volume\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.259885 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.259762    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.259885 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.259831    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:23.759815371 +0000 UTC m=+61.108297980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.260162 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.260142    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/71190e89-69d2-4d28-8825-f3de21d2de28-data-volume\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.288372 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.288344    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxdp8\" (UniqueName: \"kubernetes.io/projected/71190e89-69d2-4d28-8825-f3de21d2de28-kube-api-access-rxdp8\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.409482 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.409434    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ssnrw"
Jul 07 14:57:23.507272 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.507250    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/0.log"
Jul 07 14:57:23.507364 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.507285    2556 generic.go:356] "Generic (PLEG): container finished" podID="77040db7-a4f5-4195-8504-062e41f25c55" containerID="07508e82990a6f881b6c115275413cda9ac430ad6a5272a30342cf7d50be8746" exitCode=255
Jul 07 14:57:23.507364 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.507317    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" event={"ID":"77040db7-a4f5-4195-8504-062e41f25c55","Type":"ContainerDied","Data":"07508e82990a6f881b6c115275413cda9ac430ad6a5272a30342cf7d50be8746"}
Jul 07 14:57:23.507584 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.507569    2556 scope.go:117] "RemoveContainer" containerID="07508e82990a6f881b6c115275413cda9ac430ad6a5272a30342cf7d50be8746"
Jul 07 14:57:23.765598 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.765573    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.765745 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.765724    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.765840 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:23.765746    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:23.765840 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.765816    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:24.765773831 +0000 UTC m=+62.114256447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.765840 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.765828    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:23.766005 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:23.765867    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:24.765854434 +0000 UTC m=+62.114337041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:24.511998 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.511973    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 14:57:24.512410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.512394    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/0.log"
Jul 07 14:57:24.512456 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.512429    2556 generic.go:356] "Generic (PLEG): container finished" podID="77040db7-a4f5-4195-8504-062e41f25c55" containerID="7469bed4088fef7a96482003ccfbc213457a69b82a1a1f121fa5ab2fe1342433" exitCode=255
Jul 07 14:57:24.512510 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.512493    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" event={"ID":"77040db7-a4f5-4195-8504-062e41f25c55","Type":"ContainerDied","Data":"7469bed4088fef7a96482003ccfbc213457a69b82a1a1f121fa5ab2fe1342433"}
Jul 07 14:57:24.512557 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.512544    2556 scope.go:117] "RemoveContainer" containerID="07508e82990a6f881b6c115275413cda9ac430ad6a5272a30342cf7d50be8746"
Jul 07 14:57:24.512729 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.512710    2556 scope.go:117] "RemoveContainer" containerID="7469bed4088fef7a96482003ccfbc213457a69b82a1a1f121fa5ab2fe1342433"
Jul 07 14:57:24.512918 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:24.512899    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-k8r9l_openshift-console-operator(77040db7-a4f5-4195-8504-062e41f25c55)\"" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" podUID="77040db7-a4f5-4195-8504-062e41f25c55"
Jul 07 14:57:24.773330 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.773259    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:24.773456 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:24.773395    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:24.773456 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.773434    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:24.773456 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:24.773449    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:26.773432284 +0000 UTC m=+64.121914896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:24.773556 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:24.773476    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:24.773556 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:24.773525    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:26.773508725 +0000 UTC m=+64.121991342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:24.856657 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:24.856630    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x4xx9_0b19a2c4-299a-4d9e-a470-50b1d292899f/dns-node-resolver/0.log"
Jul 07 14:57:25.256205 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:25.256175    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9q9q5_91074283-7c9c-4a20-9196-2d55a7bc6ae1/node-ca/0.log"
Jul 07 14:57:25.519579 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:25.519517    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 14:57:25.519925 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:25.519816    2556 scope.go:117] "RemoveContainer" containerID="7469bed4088fef7a96482003ccfbc213457a69b82a1a1f121fa5ab2fe1342433"
Jul 07 14:57:25.519969 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:25.519950    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-k8r9l_openshift-console-operator(77040db7-a4f5-4195-8504-062e41f25c55)\"" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" podUID="77040db7-a4f5-4195-8504-062e41f25c55"
Jul 07 14:57:25.882833 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:25.882744    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:25.882970 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:25.882867    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:25.882970 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:25.882929    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls podName:0227a3b2-bc6c-4274-91f5-5dee553832d3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:33.882913745 +0000 UTC m=+71.231396352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-pvr7j" (UID: "0227a3b2-bc6c-4274-91f5-5dee553832d3") : secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:25.983314 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:25.983279    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:25.983447 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:25.983329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:25.983447 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:25.983422    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:57:25.983544 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:25.983483    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:33.983466612 +0000 UTC m=+71.331949242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : secret "router-metrics-certs-default" not found
Jul 07 14:57:25.983544 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:25.983504    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:33.98348978 +0000 UTC m=+71.331972387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : configmap references non-existent config key: service-ca.crt
Jul 07 14:57:26.059259 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:26.059234    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dht8q_c0c5e587-0e58-4aa8-b704-350d87c04978/serve-healthcheck-canary/0.log"
Jul 07 14:57:26.790480 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:26.790447    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:26.790849 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:26.790557    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:26.790849 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:26.790577    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:26.790849 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:26.790627    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:30.790614928 +0000 UTC m=+68.139097535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:26.790849 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:26.790631    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:26.790849 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:26.790672    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:30.790661042 +0000 UTC m=+68.139143650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:26.891205 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:26.891179    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:26.891318 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:26.891300    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:57:26.891370 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:26.891351    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls podName:6b491dac-474d-4cc9-9ba3-c9eca1a04b5e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:34.89134013 +0000 UTC m=+72.239822737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zsr8" (UID: "6b491dac-474d-4cc9-9ba3-c9eca1a04b5e") : secret "samples-operator-tls" not found
Jul 07 14:57:28.000616 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.000553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:57:28.001073 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.000620    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:57:28.001073 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.000678    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:57:28.003113 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.003090    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:57:28.003228 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.003121    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:57:28.003228 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.003148    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:57:28.011309 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:28.011290    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 14:57:28.011371 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:28.011346    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs podName:b6fd483f-453e-4289-82fb-02ad4d241017 nodeName:}" failed. No retries permitted until 2026-07-07 14:58:32.011328606 +0000 UTC m=+129.359811214 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs") pod "network-metrics-daemon-wxd9k" (UID: "b6fd483f-453e-4289-82fb-02ad4d241017") : secret "metrics-daemon-secret" not found
Jul 07 14:57:28.013334 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.013313    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:57:28.013448 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.013419    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dec33f92-47f6-4040-8a3b-4658fdbf72ca-original-pull-secret\") pod \"global-pull-secret-syncer-f7mm6\" (UID: \"dec33f92-47f6-4040-8a3b-4658fdbf72ca\") " pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:57:28.023438 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.023418    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv4c5\" (UniqueName: \"kubernetes.io/projected/70153084-d026-4d8c-a91c-dff91ff1f515-kube-api-access-wv4c5\") pod \"network-check-target-756dp\" (UID: \"70153084-d026-4d8c-a91c-dff91ff1f515\") " pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:57:28.086061 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.086041    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jrv9k\""
Jul 07 14:57:28.094158 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.094137    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:57:28.215540 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.215516    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-756dp"]
Jul 07 14:57:28.221224 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:28.221195    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70153084_d026_4d8c_a91c_dff91ff1f515.slice/crio-0aae990073d3f25f2d5f906a4a53bd24b50471162c4396051e41fedea00ea883 WatchSource:0}: Error finding container 0aae990073d3f25f2d5f906a4a53bd24b50471162c4396051e41fedea00ea883: Status 404 returned error can't find the container with id 0aae990073d3f25f2d5f906a4a53bd24b50471162c4396051e41fedea00ea883
Jul 07 14:57:28.290001 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.289949    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7mm6"
Jul 07 14:57:28.404961 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.404838    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f7mm6"]
Jul 07 14:57:28.407530 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:28.407507    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec33f92_47f6_4040_8a3b_4658fdbf72ca.slice/crio-f991e216086e2022d1c40a57747512604bc47dccea6922d42c8060a545ffaac6 WatchSource:0}: Error finding container f991e216086e2022d1c40a57747512604bc47dccea6922d42c8060a545ffaac6: Status 404 returned error can't find the container with id f991e216086e2022d1c40a57747512604bc47dccea6922d42c8060a545ffaac6
Jul 07 14:57:28.528154 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.528128    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-756dp" event={"ID":"70153084-d026-4d8c-a91c-dff91ff1f515","Type":"ContainerStarted","Data":"0aae990073d3f25f2d5f906a4a53bd24b50471162c4396051e41fedea00ea883"}
Jul 07 14:57:28.529227 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:28.529207    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f7mm6" event={"ID":"dec33f92-47f6-4040-8a3b-4658fdbf72ca","Type":"ContainerStarted","Data":"f991e216086e2022d1c40a57747512604bc47dccea6922d42c8060a545ffaac6"}
Jul 07 14:57:29.916492 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:29.916458    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:57:29.916952 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:29.916567    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:57:29.916952 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:29.916635    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:29.916952 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:29.916655    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78c556786f-mfrd7: secret "image-registry-tls" not found
Jul 07 14:57:29.916952 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:29.916681    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:29.916952 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:29.916724    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls podName:012cad90-75d4-490c-82fe-fba736164700 nodeName:}" failed. No retries permitted until 2026-07-07 14:58:01.916701751 +0000 UTC m=+99.265184361 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls") pod "image-registry-78c556786f-mfrd7" (UID: "012cad90-75d4-490c-82fe-fba736164700") : secret "image-registry-tls" not found
Jul 07 14:57:29.916952 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:29.916741    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls podName:0718570a-e9fe-460b-a0c9-b895125c2d44 nodeName:}" failed. No retries permitted until 2026-07-07 14:58:01.916732417 +0000 UTC m=+99.265215054 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls") pod "dns-default-xjl76" (UID: "0718570a-e9fe-460b-a0c9-b895125c2d44") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:30.397907 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:30.397822    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:30.397907 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:30.397888    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:30.398247 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:30.398233    2556 scope.go:117] "RemoveContainer" containerID="7469bed4088fef7a96482003ccfbc213457a69b82a1a1f121fa5ab2fe1342433"
Jul 07 14:57:30.398424 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:30.398396    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-k8r9l_openshift-console-operator(77040db7-a4f5-4195-8504-062e41f25c55)\"" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" podUID="77040db7-a4f5-4195-8504-062e41f25c55"
Jul 07 14:57:30.825876 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:30.825828    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:30.826059 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:30.825961    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:30.826059 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:30.826050    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.826177 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:30.826112    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:38.826095149 +0000 UTC m=+76.174577762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.826459 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:30.826366    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:30.826459 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:30.826438    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:38.826417206 +0000 UTC m=+76.174899823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:31.540493 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:31.540458    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-756dp" event={"ID":"70153084-d026-4d8c-a91c-dff91ff1f515","Type":"ContainerStarted","Data":"e64a25050dc12d44f7f257e7d19938d85f04821506eee2036d21b49e70fb3bd7"}
Jul 07 14:57:31.541066 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:31.541045    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:57:31.558926 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:31.558878    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-756dp" podStartSLOduration=65.714493084 podStartE2EDuration="1m8.558865849s" totalImagesPullingTime="2.844372765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:31.557390286 +0000 UTC m=+68.905872917" watchObservedRunningTime="2026-07-07 14:57:31.558865849 +0000 UTC m=+68.907348479"
Jul 07 14:57:33.547923 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:33.547879    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f7mm6" event={"ID":"dec33f92-47f6-4040-8a3b-4658fdbf72ca","Type":"ContainerStarted","Data":"b00940cd11316f3fbe1efff0d57231c3e6bdf0c1c7714f00b8e25a8bcb64bdc4"}
Jul 07 14:57:33.564476 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:33.564418    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f7mm6" podStartSLOduration=65.131866993 podStartE2EDuration="1m9.564397455s" totalImagesPullingTime="4.432530462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:33.563862536 +0000 UTC m=+70.912345183" watchObservedRunningTime="2026-07-07 14:57:33.564397455 +0000 UTC m=+70.912880145"
Jul 07 14:57:33.950445 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:33.950410    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:33.950614 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:33.950570    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:33.950669 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:33.950643    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls podName:0227a3b2-bc6c-4274-91f5-5dee553832d3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:49.950625558 +0000 UTC m=+87.299108165 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-pvr7j" (UID: "0227a3b2-bc6c-4274-91f5-5dee553832d3") : secret "cluster-monitoring-operator-tls" not found
Jul 07 14:57:34.051830 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:34.051784    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:34.051996 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:34.051850    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:34.051996 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:34.051954    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:57:34.052124 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:34.051963    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:50.051950035 +0000 UTC m=+87.400432642 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : configmap references non-existent config key: service-ca.crt
Jul 07 14:57:34.052124 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:34.052115    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs podName:272f03a0-8b74-47c6-b8f5-7d82a5d4bba8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:50.052097389 +0000 UTC m=+87.400580002 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs") pod "router-default-5f8f66bc9c-fhp6v" (UID: "272f03a0-8b74-47c6-b8f5-7d82a5d4bba8") : secret "router-metrics-certs-default" not found
Jul 07 14:57:34.959315 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:34.959289    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:34.959666 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:34.959403    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:57:34.959666 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:34.959449    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls podName:6b491dac-474d-4cc9-9ba3-c9eca1a04b5e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:50.959437653 +0000 UTC m=+88.307920260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7zsr8" (UID: "6b491dac-474d-4cc9-9ba3-c9eca1a04b5e") : secret "samples-operator-tls" not found
Jul 07 14:57:38.888683 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:38.888642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:38.889118 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:38.888714    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:38.889118 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:38.888832    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:38.889118 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:57:38.888904    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls podName:71190e89-69d2-4d28-8825-f3de21d2de28 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:54.888888666 +0000 UTC m=+92.237371273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mb4h" (UID: "71190e89-69d2-4d28-8825-f3de21d2de28") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:38.889289 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:38.889262    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/71190e89-69d2-4d28-8825-f3de21d2de28-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:43.266153 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:43.266126    2556 scope.go:117] "RemoveContainer" containerID="7469bed4088fef7a96482003ccfbc213457a69b82a1a1f121fa5ab2fe1342433"
Jul 07 14:57:43.578895 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:43.578835    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 14:57:43.579014 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:43.578908    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" event={"ID":"77040db7-a4f5-4195-8504-062e41f25c55","Type":"ContainerStarted","Data":"dcf00051478b51628cc77e507cc23af8c5afe1ddff48afab1e8d25dc7fe480f1"}
Jul 07 14:57:43.579267 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:43.579242    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:43.597786 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:43.597738    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l" podStartSLOduration=20.753453664 podStartE2EDuration="23.597726835s" totalImagesPullingTime="2.844273171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:43.596978043 +0000 UTC m=+80.945460701" watchObservedRunningTime="2026-07-07 14:57:43.597726835 +0000 UTC m=+80.946209470"
Jul 07 14:57:43.742508 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:43.742482    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-k8r9l"
Jul 07 14:57:49.976100 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:49.975964    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:49.978648 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:49.978608    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0227a3b2-bc6c-4274-91f5-5dee553832d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-pvr7j\" (UID: \"0227a3b2-bc6c-4274-91f5-5dee553832d3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:50.076609 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.076576    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:50.076739 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.076627    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:50.077337 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.077315    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-service-ca-bundle\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:50.079146 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.079124    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/272f03a0-8b74-47c6-b8f5-7d82a5d4bba8-metrics-certs\") pod \"router-default-5f8f66bc9c-fhp6v\" (UID: \"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8\") " pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:50.240495 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.240431    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ggbvw\""
Jul 07 14:57:50.248797 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.248775    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"
Jul 07 14:57:50.343087 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.343057    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bl5fp\""
Jul 07 14:57:50.350602 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.350517    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:50.400745 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.400714    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j"]
Jul 07 14:57:50.481825 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.481774    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f8f66bc9c-fhp6v"]
Jul 07 14:57:50.483606 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:50.483584    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272f03a0_8b74_47c6_b8f5_7d82a5d4bba8.slice/crio-4035570ff494d0fe0720aa9daed40c3b4cfbe20a24ac422cb04c01c1f1a52ce6 WatchSource:0}: Error finding container 4035570ff494d0fe0720aa9daed40c3b4cfbe20a24ac422cb04c01c1f1a52ce6: Status 404 returned error can't find the container with id 4035570ff494d0fe0720aa9daed40c3b4cfbe20a24ac422cb04c01c1f1a52ce6
Jul 07 14:57:50.601551 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.601516    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j" event={"ID":"0227a3b2-bc6c-4274-91f5-5dee553832d3","Type":"ContainerStarted","Data":"92a2d9393a02197c0df81397b012a12b04850bcd16cea2f84065909370f72bcb"}
Jul 07 14:57:50.602850 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.602832    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v" event={"ID":"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8","Type":"ContainerStarted","Data":"9700804ced39e79cae24e8411a1b0882fd32fe172c577afa25a2bffb7f0a5d5c"}
Jul 07 14:57:50.602941 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.602867    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v" event={"ID":"272f03a0-8b74-47c6-b8f5-7d82a5d4bba8","Type":"ContainerStarted","Data":"4035570ff494d0fe0720aa9daed40c3b4cfbe20a24ac422cb04c01c1f1a52ce6"}
Jul 07 14:57:50.625232 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.625169    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v" podStartSLOduration=32.625158082 podStartE2EDuration="32.625158082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:50.624667582 +0000 UTC m=+87.973150211" watchObservedRunningTime="2026-07-07 14:57:50.625158082 +0000 UTC m=+87.973640732"
Jul 07 14:57:50.985643 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.985616    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:50.987949 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:50.987926    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b491dac-474d-4cc9-9ba3-c9eca1a04b5e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7zsr8\" (UID: \"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:51.246033 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.245968    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gbm9l\""
Jul 07 14:57:51.254062 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.254042    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"
Jul 07 14:57:51.351375 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.351226    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:51.354372 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.354350    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:51.391249 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.391218    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8"]
Jul 07 14:57:51.610622 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.610550    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8" event={"ID":"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e","Type":"ContainerStarted","Data":"49fdf8f322ac23cbab4207538aa389a629236d8f86910460f0c26f384b6c2bb5"}
Jul 07 14:57:51.610917 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.610896    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:51.612140 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:51.612114    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5f8f66bc9c-fhp6v"
Jul 07 14:57:52.615858 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:52.615819    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j" event={"ID":"0227a3b2-bc6c-4274-91f5-5dee553832d3","Type":"ContainerStarted","Data":"3bc3486385ad7a46b87c9c9a26bee7c2829896c41ae7fd9c009faf10acb53a9c"}
Jul 07 14:57:52.635051 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:52.634994    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-pvr7j" podStartSLOduration=32.631085271 podStartE2EDuration="34.634972974s" totalImagesPullingTime="2.003887703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:52.634061713 +0000 UTC m=+89.982544369" watchObservedRunningTime="2026-07-07 14:57:52.634972974 +0000 UTC m=+89.983455608"
Jul 07 14:57:53.621117 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.621073    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8" event={"ID":"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e","Type":"ContainerStarted","Data":"778a48f5ab6b95d87d595576646ad97a7f09119bd4cceb562c883929e56424e4"}
Jul 07 14:57:53.621606 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.621134    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8" event={"ID":"6b491dac-474d-4cc9-9ba3-c9eca1a04b5e","Type":"ContainerStarted","Data":"3b63cdd8383c27d16d247a3d11838d5b41aff7c4a531741cf57d11cd32ef6474"}
Jul 07 14:57:53.645023 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.644975    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7zsr8" podStartSLOduration=33.111149076 podStartE2EDuration="34.644958755s" totalImagesPullingTime="1.533809679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:53.643332899 +0000 UTC m=+90.991815546" watchObservedRunningTime="2026-07-07 14:57:53.644958755 +0000 UTC m=+90.993441387"
Jul 07 14:57:53.820057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.820028    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-php5t"]
Jul 07 14:57:53.823363 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.823347    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-php5t"
Jul 07 14:57:53.828989 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.828967    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"]
Jul 07 14:57:53.831665 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.831635    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 14:57:53.831846 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.831828    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gbg2d\""
Jul 07 14:57:53.831916 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.831873    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 14:57:53.832090 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.832074    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"]
Jul 07 14:57:53.832199 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.832186    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"
Jul 07 14:57:53.834903 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.834881    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5888f9f647-q8kr7"]
Jul 07 14:57:53.835004 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.834976    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:53.836393 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.835460    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 07 14:57:53.836393 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.835476    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f664q\""
Jul 07 14:57:53.839077 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.838865    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-php5t"]
Jul 07 14:57:53.839077 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.838965    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:53.839613 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.839363    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-567gv\""
Jul 07 14:57:53.839613 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.839497    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 14:57:53.840125 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.840105    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 14:57:53.842005 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.841835    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 14:57:53.842005 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.841835    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 14:57:53.842005 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.841836    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 14:57:53.842387 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.842324    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 14:57:53.842500 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.842478    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dmhw2\""
Jul 07 14:57:53.842570 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.842548    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 14:57:53.851914 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.851888    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"]
Jul 07 14:57:53.852912 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.852892    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"]
Jul 07 14:57:53.853862 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.853843    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5888f9f647-q8kr7"]
Jul 07 14:57:53.910058 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910005    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-oauth-serving-cert\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:53.910058 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-service-ca\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:53.910058 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910054    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-config\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:53.910213 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910074    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-oauth-config\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:53.910213 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910144    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpnd9\" (UniqueName: \"kubernetes.io/projected/b1c843ae-67f8-4d36-80cc-6fb63018755a-kube-api-access-kpnd9\") pod \"downloads-685bfb5f64-php5t\" (UID: \"b1c843ae-67f8-4d36-80cc-6fb63018755a\") " pod="openshift-console/downloads-685bfb5f64-php5t"
Jul 07 14:57:53.910213 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910162    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1aced028-98bf-45c7-a2e6-69e6476d4b0d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-knkrb\" (UID: \"1aced028-98bf-45c7-a2e6-69e6476d4b0d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"
Jul 07 14:57:53.910546 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910185    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7b518f69-69ee-4d12-b33e-d4a7d4792b55-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qnkcv\" (UID: \"7b518f69-69ee-4d12-b33e-d4a7d4792b55\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:53.910617 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910584    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-serving-cert\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:53.910670 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910626    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b518f69-69ee-4d12-b33e-d4a7d4792b55-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qnkcv\" (UID: \"7b518f69-69ee-4d12-b33e-d4a7d4792b55\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:53.911043 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:53.910662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6tv7\" (UniqueName: \"kubernetes.io/projected/4bdff553-4c5c-40be-ab27-d98f6af894ee-kube-api-access-x6tv7\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.011993 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.011970    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpnd9\" (UniqueName: \"kubernetes.io/projected/b1c843ae-67f8-4d36-80cc-6fb63018755a-kube-api-access-kpnd9\") pod \"downloads-685bfb5f64-php5t\" (UID: \"b1c843ae-67f8-4d36-80cc-6fb63018755a\") " pod="openshift-console/downloads-685bfb5f64-php5t"
Jul 07 14:57:54.012116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012005    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1aced028-98bf-45c7-a2e6-69e6476d4b0d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-knkrb\" (UID: \"1aced028-98bf-45c7-a2e6-69e6476d4b0d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"
Jul 07 14:57:54.012116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012033    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7b518f69-69ee-4d12-b33e-d4a7d4792b55-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qnkcv\" (UID: \"7b518f69-69ee-4d12-b33e-d4a7d4792b55\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:54.012116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-serving-cert\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.012116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012084    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b518f69-69ee-4d12-b33e-d4a7d4792b55-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qnkcv\" (UID: \"7b518f69-69ee-4d12-b33e-d4a7d4792b55\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:54.012351 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012306    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6tv7\" (UniqueName: \"kubernetes.io/projected/4bdff553-4c5c-40be-ab27-d98f6af894ee-kube-api-access-x6tv7\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.012469 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012447    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-oauth-serving-cert\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.012614 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012595    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-service-ca\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.012779 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012765    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-config\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.012938 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012916    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-oauth-config\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.013201 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.012820    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b518f69-69ee-4d12-b33e-d4a7d4792b55-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qnkcv\" (UID: \"7b518f69-69ee-4d12-b33e-d4a7d4792b55\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:54.013385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.013360    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-service-ca\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.013474 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.013440    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-config\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.013474 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.013198    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-oauth-serving-cert\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.014703 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.014676    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7b518f69-69ee-4d12-b33e-d4a7d4792b55-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qnkcv\" (UID: \"7b518f69-69ee-4d12-b33e-d4a7d4792b55\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:54.014780 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.014743    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1aced028-98bf-45c7-a2e6-69e6476d4b0d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-knkrb\" (UID: \"1aced028-98bf-45c7-a2e6-69e6476d4b0d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"
Jul 07 14:57:54.014849 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.014784    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-serving-cert\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.014954 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.014939    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-oauth-config\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.022484 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.022460    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6tv7\" (UniqueName: \"kubernetes.io/projected/4bdff553-4c5c-40be-ab27-d98f6af894ee-kube-api-access-x6tv7\") pod \"console-5888f9f647-q8kr7\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") " pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.025718 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.025694    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpnd9\" (UniqueName: \"kubernetes.io/projected/b1c843ae-67f8-4d36-80cc-6fb63018755a-kube-api-access-kpnd9\") pod \"downloads-685bfb5f64-php5t\" (UID: \"b1c843ae-67f8-4d36-80cc-6fb63018755a\") " pod="openshift-console/downloads-685bfb5f64-php5t"
Jul 07 14:57:54.134013 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.133993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-php5t"
Jul 07 14:57:54.145260 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.145238    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"
Jul 07 14:57:54.152607 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.152588    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"
Jul 07 14:57:54.159362 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.159323    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:57:54.304553 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.304527    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-php5t"]
Jul 07 14:57:54.312526 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:54.312500    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c843ae_67f8_4d36_80cc_6fb63018755a.slice/crio-aa4323dc677c28bc0e41b40754d818d30bcfbc66441d2df54467a7a4639be1a6 WatchSource:0}: Error finding container aa4323dc677c28bc0e41b40754d818d30bcfbc66441d2df54467a7a4639be1a6: Status 404 returned error can't find the container with id aa4323dc677c28bc0e41b40754d818d30bcfbc66441d2df54467a7a4639be1a6
Jul 07 14:57:54.413173 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.413150    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5888f9f647-q8kr7"]
Jul 07 14:57:54.415690 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:54.415667    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdff553_4c5c_40be_ab27_d98f6af894ee.slice/crio-5107f79a751f1f400a0fcf64445778c46650762d6981803e1a88000ac6242f53 WatchSource:0}: Error finding container 5107f79a751f1f400a0fcf64445778c46650762d6981803e1a88000ac6242f53: Status 404 returned error can't find the container with id 5107f79a751f1f400a0fcf64445778c46650762d6981803e1a88000ac6242f53
Jul 07 14:57:54.520141 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.520122    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv"]
Jul 07 14:57:54.522199 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:54.522170    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b518f69_69ee_4d12_b33e_d4a7d4792b55.slice/crio-4de93d4799217e40dce030fe6568528041dd6d11b86db1362d971bbd6a985635 WatchSource:0}: Error finding container 4de93d4799217e40dce030fe6568528041dd6d11b86db1362d971bbd6a985635: Status 404 returned error can't find the container with id 4de93d4799217e40dce030fe6568528041dd6d11b86db1362d971bbd6a985635
Jul 07 14:57:54.524887 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.524866    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"]
Jul 07 14:57:54.526910 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:54.526892    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aced028_98bf_45c7_a2e6_69e6476d4b0d.slice/crio-0806ac0f4ed698da8c3f9f12a60794fb15743406b966d65ad74cd022d6877652 WatchSource:0}: Error finding container 0806ac0f4ed698da8c3f9f12a60794fb15743406b966d65ad74cd022d6877652: Status 404 returned error can't find the container with id 0806ac0f4ed698da8c3f9f12a60794fb15743406b966d65ad74cd022d6877652
Jul 07 14:57:54.625564 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.625536    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5888f9f647-q8kr7" event={"ID":"4bdff553-4c5c-40be-ab27-d98f6af894ee","Type":"ContainerStarted","Data":"5107f79a751f1f400a0fcf64445778c46650762d6981803e1a88000ac6242f53"}
Jul 07 14:57:54.626620 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.626599    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb" event={"ID":"1aced028-98bf-45c7-a2e6-69e6476d4b0d","Type":"ContainerStarted","Data":"0806ac0f4ed698da8c3f9f12a60794fb15743406b966d65ad74cd022d6877652"}
Jul 07 14:57:54.627707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.627690    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-php5t" event={"ID":"b1c843ae-67f8-4d36-80cc-6fb63018755a","Type":"ContainerStarted","Data":"aa4323dc677c28bc0e41b40754d818d30bcfbc66441d2df54467a7a4639be1a6"}
Jul 07 14:57:54.628727 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.628708    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv" event={"ID":"7b518f69-69ee-4d12-b33e-d4a7d4792b55","Type":"ContainerStarted","Data":"4de93d4799217e40dce030fe6568528041dd6d11b86db1362d971bbd6a985635"}
Jul 07 14:57:54.919046 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.919022    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:54.921048 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:54.921025    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/71190e89-69d2-4d28-8825-f3de21d2de28-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mb4h\" (UID: \"71190e89-69d2-4d28-8825-f3de21d2de28\") " pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:55.170660 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:55.170588    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mb4h"
Jul 07 14:57:55.343924 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:55.343850    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mb4h"]
Jul 07 14:57:55.346681 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:57:55.346651    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71190e89_69d2_4d28_8825_f3de21d2de28.slice/crio-06c5350e8da92654145d278b26e77c78bac9fc8a08cf8798e38480c2f0f56db3 WatchSource:0}: Error finding container 06c5350e8da92654145d278b26e77c78bac9fc8a08cf8798e38480c2f0f56db3: Status 404 returned error can't find the container with id 06c5350e8da92654145d278b26e77c78bac9fc8a08cf8798e38480c2f0f56db3
Jul 07 14:57:55.635986 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:55.635859    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mb4h" event={"ID":"71190e89-69d2-4d28-8825-f3de21d2de28","Type":"ContainerStarted","Data":"06c5350e8da92654145d278b26e77c78bac9fc8a08cf8798e38480c2f0f56db3"}
Jul 07 14:57:57.647141 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:57.647094    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb" event={"ID":"1aced028-98bf-45c7-a2e6-69e6476d4b0d","Type":"ContainerStarted","Data":"e28b3c6a64db977ee1e30c120d044d04263f17a8768312b562b05f376e669e2e"}
Jul 07 14:57:57.647587 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:57.647532    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"
Jul 07 14:57:57.649290 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:57.649258    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv" event={"ID":"7b518f69-69ee-4d12-b33e-d4a7d4792b55","Type":"ContainerStarted","Data":"79afa3d06d2e5e9743e15398de8a2fed05391bb39d91c64b5ec4dd4c6aa46835"}
Jul 07 14:57:57.653771 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:57.653746    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb"
Jul 07 14:57:57.666975 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:57.666926    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-knkrb" podStartSLOduration=2.2338202369999998 podStartE2EDuration="4.666909775s" totalImagesPullingTime="2.433089538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:57.666077663 +0000 UTC m=+95.014560294" watchObservedRunningTime="2026-07-07 14:57:57.666909775 +0000 UTC m=+95.015392385"
Jul 07 14:57:57.687589 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:57.687543    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qnkcv" podStartSLOduration=2.248261135 podStartE2EDuration="4.687530084s" totalImagesPullingTime="2.439268949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:57.687278917 +0000 UTC m=+95.035761547" watchObservedRunningTime="2026-07-07 14:57:57.687530084 +0000 UTC m=+95.036012714"
Jul 07 14:57:58.437762 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.437729    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f75dffcf9-z7m8p"]
Jul 07 14:57:58.446561 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.446535    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.455186 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.455048    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 14:57:58.455410 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.455357    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f75dffcf9-z7m8p"]
Jul 07 14:57:58.552308 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.552275    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-oauth-serving-cert\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.552476 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.552332    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-serving-cert\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.552476 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.552367    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-config\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.552476 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.552383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-oauth-config\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.552476 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.552413    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-service-ca\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.552677 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.552486    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-trusted-ca-bundle\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.552677 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.552527    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwrv6\" (UniqueName: \"kubernetes.io/projected/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-kube-api-access-zwrv6\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.653355 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-service-ca\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.653355 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-trusted-ca-bundle\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.653882 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653363    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwrv6\" (UniqueName: \"kubernetes.io/projected/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-kube-api-access-zwrv6\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.653882 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-oauth-serving-cert\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.653882 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653444    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-serving-cert\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.653882 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653480    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-config\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.653882 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653524    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-oauth-config\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.654145 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.653967    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-service-ca\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.654207 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.654171    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-oauth-serving-cert\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.654525 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.654500    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-config\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.655266 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.655190    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-trusted-ca-bundle\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.656435 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.656414    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-oauth-config\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.656914 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.656898    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-serving-cert\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.662811 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.662773    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwrv6\" (UniqueName: \"kubernetes.io/projected/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-kube-api-access-zwrv6\") pod \"console-f75dffcf9-z7m8p\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") " pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:57:58.758645 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:57:58.758576    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:58:01.084949 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.084924    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f75dffcf9-z7m8p"]
Jul 07 14:58:01.089366 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:01.089335    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0c2758_cf78_4673_be66_fee1ba7ff1d5.slice/crio-4d58c4f4efae0f5f870a9a5f5635a0808d6380f7f44bf53719b9d66a05b5dfab WatchSource:0}: Error finding container 4d58c4f4efae0f5f870a9a5f5635a0808d6380f7f44bf53719b9d66a05b5dfab: Status 404 returned error can't find the container with id 4d58c4f4efae0f5f870a9a5f5635a0808d6380f7f44bf53719b9d66a05b5dfab
Jul 07 14:58:01.665911 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.665874    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5888f9f647-q8kr7" event={"ID":"4bdff553-4c5c-40be-ab27-d98f6af894ee","Type":"ContainerStarted","Data":"489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff"}
Jul 07 14:58:01.667585 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.667559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mb4h" event={"ID":"71190e89-69d2-4d28-8825-f3de21d2de28","Type":"ContainerStarted","Data":"732f433fa4e130ccf1bb344f82988b825e28c08e86b0cdc7058306b7452fb192"}
Jul 07 14:58:01.669171 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.669144    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75dffcf9-z7m8p" event={"ID":"ec0c2758-cf78-4673-be66-fee1ba7ff1d5","Type":"ContainerStarted","Data":"8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4"}
Jul 07 14:58:01.669292 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.669206    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75dffcf9-z7m8p" event={"ID":"ec0c2758-cf78-4673-be66-fee1ba7ff1d5","Type":"ContainerStarted","Data":"4d58c4f4efae0f5f870a9a5f5635a0808d6380f7f44bf53719b9d66a05b5dfab"}
Jul 07 14:58:01.687691 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.687648    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5888f9f647-q8kr7" podStartSLOduration=2.166016759 podStartE2EDuration="8.687628331s" totalImagesPullingTime="6.521611572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:01.687233811 +0000 UTC m=+99.035716440" watchObservedRunningTime="2026-07-07 14:58:01.687628331 +0000 UTC m=+99.036110962"
Jul 07 14:58:01.711036 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.710919    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f75dffcf9-z7m8p" podStartSLOduration=3.710900869 podStartE2EDuration="3.710900869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:01.710230615 +0000 UTC m=+99.058713263" watchObservedRunningTime="2026-07-07 14:58:01.710900869 +0000 UTC m=+99.059383499"
Jul 07 14:58:01.985089 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.985061    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:58:01.985247 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.985204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:58:01.987739 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.987717    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0718570a-e9fe-460b-a0c9-b895125c2d44-metrics-tls\") pod \"dns-default-xjl76\" (UID: \"0718570a-e9fe-460b-a0c9-b895125c2d44\") " pod="openshift-dns/dns-default-xjl76"
Jul 07 14:58:01.987877 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:01.987857    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"image-registry-78c556786f-mfrd7\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") " pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:58:02.240038 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.239973    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mrpzk\""
Jul 07 14:58:02.246579 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.246559    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:58:02.271123 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.271102    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pkm7x\""
Jul 07 14:58:02.276178 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.276154    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xjl76"
Jul 07 14:58:02.449484 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.449455    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78c556786f-mfrd7"]
Jul 07 14:58:02.453147 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.453080    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xjl76"]
Jul 07 14:58:02.454643 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:02.454599    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012cad90_75d4_490c_82fe_fba736164700.slice/crio-3cbf9ad745bbb79bb0efbd982d7a1f41862629dd9cd2e671cacb077e7aaad7a5 WatchSource:0}: Error finding container 3cbf9ad745bbb79bb0efbd982d7a1f41862629dd9cd2e671cacb077e7aaad7a5: Status 404 returned error can't find the container with id 3cbf9ad745bbb79bb0efbd982d7a1f41862629dd9cd2e671cacb077e7aaad7a5
Jul 07 14:58:02.456000 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:02.455963    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0718570a_e9fe_460b_a0c9_b895125c2d44.slice/crio-0fbdda4787d0f1ba20e3c87d19d70f234a2ce2e78d0ddc61dc431591c207fe84 WatchSource:0}: Error finding container 0fbdda4787d0f1ba20e3c87d19d70f234a2ce2e78d0ddc61dc431591c207fe84: Status 404 returned error can't find the container with id 0fbdda4787d0f1ba20e3c87d19d70f234a2ce2e78d0ddc61dc431591c207fe84
Jul 07 14:58:02.546228 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.546205    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-756dp"
Jul 07 14:58:02.674095 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.674063    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xjl76" event={"ID":"0718570a-e9fe-460b-a0c9-b895125c2d44","Type":"ContainerStarted","Data":"0fbdda4787d0f1ba20e3c87d19d70f234a2ce2e78d0ddc61dc431591c207fe84"}
Jul 07 14:58:02.676240 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.676210    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mb4h" event={"ID":"71190e89-69d2-4d28-8825-f3de21d2de28","Type":"ContainerStarted","Data":"109d294b00b725e31bbd8def0d00b16f22006feddf86a9a0867f3a6656978879"}
Jul 07 14:58:02.677873 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.677845    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c556786f-mfrd7" event={"ID":"012cad90-75d4-490c-82fe-fba736164700","Type":"ContainerStarted","Data":"b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077"}
Jul 07 14:58:02.678010 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.677921    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:58:02.678010 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.677956    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c556786f-mfrd7" event={"ID":"012cad90-75d4-490c-82fe-fba736164700","Type":"ContainerStarted","Data":"3cbf9ad745bbb79bb0efbd982d7a1f41862629dd9cd2e671cacb077e7aaad7a5"}
Jul 07 14:58:02.700381 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:02.700335    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78c556786f-mfrd7" podStartSLOduration=98.700317808 podStartE2EDuration="1m38.700317808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:02.69910237 +0000 UTC m=+100.047585029" watchObservedRunningTime="2026-07-07 14:58:02.700317808 +0000 UTC m=+100.048800442"
Jul 07 14:58:04.159836 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.159783    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:58:04.160320 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.159894    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:58:04.161354 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.161321    2556 patch_prober.go:28] interesting pod/console-5888f9f647-q8kr7 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body=
Jul 07 14:58:04.161472 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.161386    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5888f9f647-q8kr7" podUID="4bdff553-4c5c-40be-ab27-d98f6af894ee" containerName="console" probeResult="failure" output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused"
Jul 07 14:58:04.424299 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.424259    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-77t6k"]
Jul 07 14:58:04.438250 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.438227    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.440961 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.440934    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 14:58:04.441161 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.440934    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 14:58:04.441274 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.441212    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 14:58:04.441762 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.441742    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ggm5j\""
Jul 07 14:58:04.443256 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.443232    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 14:58:04.510296 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510258    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-textfile\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510424 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510324    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-metrics-client-ca\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510424 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510419    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-wtmp\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510535 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510439    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510535 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510467    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-sys\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510535 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510491    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510647 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510534    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-root\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510647 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510575    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-tls\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.510647 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.510611    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrwnx\" (UniqueName: \"kubernetes.io/projected/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-kube-api-access-vrwnx\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611487 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611456    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-wtmp\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611649 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611501    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611649 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611531    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-sys\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611649 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611556    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611649 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-root\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611649 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611630    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-tls\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611938 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611651    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrwnx\" (UniqueName: \"kubernetes.io/projected/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-kube-api-access-vrwnx\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611938 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611694    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-textfile\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.611938 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.611747    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-metrics-client-ca\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.612483    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-metrics-client-ca\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.612532    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.612564    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-root\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.612600    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-sys\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:58:04.612651    2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:58:04.612703    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-tls podName:e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e nodeName:}" failed. No retries permitted until 2026-07-07 14:58:05.112686242 +0000 UTC m=+102.461168854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-tls") pod "node-exporter-77t6k" (UID: "e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e") : secret "node-exporter-tls" not found
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.613011    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-textfile\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.613053 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.613043    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-wtmp\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.615509 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.615483    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:04.623304 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:04.623282    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrwnx\" (UniqueName: \"kubernetes.io/projected/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-kube-api-access-vrwnx\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:05.118683 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.118650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-tls\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:05.121371 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.121343    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e-node-exporter-tls\") pod \"node-exporter-77t6k\" (UID: \"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e\") " pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:05.349658 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.349501    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-77t6k"
Jul 07 14:58:05.576670 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.576634    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 14:58:05.580157 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.580121    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.584006 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.583923    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 07 14:58:05.586153 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.585545    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 07 14:58:05.586153 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.585776    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 07 14:58:05.586153 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.586012    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 07 14:58:05.586360 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.586298    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nmwr9\""
Jul 07 14:58:05.589588 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.589569    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 07 14:58:05.589906 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.589814    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 07 14:58:05.590051 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.589940    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 07 14:58:05.590051 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.589576    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 07 14:58:05.590192 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.590174    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 07 14:58:05.606418 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.606393    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 14:58:05.723296 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723271    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9b4j\" (UniqueName: \"kubernetes.io/projected/23ff7c29-eab4-46c8-831f-8aa51b684134-kube-api-access-m9b4j\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723296 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723301    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-web-config\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723510 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723510 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723372    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23ff7c29-eab4-46c8-831f-8aa51b684134-config-out\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723510 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723510 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723457    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723510 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723505    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723751 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723563    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23ff7c29-eab4-46c8-831f-8aa51b684134-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723751 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723595    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23ff7c29-eab4-46c8-831f-8aa51b684134-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723751 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723622    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723751 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23ff7c29-eab4-46c8-831f-8aa51b684134-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723751 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723689    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-config-volume\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.723751 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.723712    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23ff7c29-eab4-46c8-831f-8aa51b684134-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.824887 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.824849    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-web-config\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825048 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.824904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825048 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.824943    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23ff7c29-eab4-46c8-831f-8aa51b684134-config-out\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825048 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.824973    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825048 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825011    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825048 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825095    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23ff7c29-eab4-46c8-831f-8aa51b684134-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23ff7c29-eab4-46c8-831f-8aa51b684134-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825151    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825194    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23ff7c29-eab4-46c8-831f-8aa51b684134-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825220    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-config-volume\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825243    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23ff7c29-eab4-46c8-831f-8aa51b684134-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.825321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9b4j\" (UniqueName: \"kubernetes.io/projected/23ff7c29-eab4-46c8-831f-8aa51b684134-kube-api-access-m9b4j\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.826300 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.825988    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23ff7c29-eab4-46c8-831f-8aa51b684134-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.826421 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.826377    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23ff7c29-eab4-46c8-831f-8aa51b684134-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.827188 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.827071    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23ff7c29-eab4-46c8-831f-8aa51b684134-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.828363 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.828308    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.828857 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.828771    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.829327 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.829062    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.829327 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.829074    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-web-config\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.829982 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.829896    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.830199 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.830176    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23ff7c29-eab4-46c8-831f-8aa51b684134-config-out\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.830332 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.830309    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-config-volume\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.830388 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.830322    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23ff7c29-eab4-46c8-831f-8aa51b684134-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.831614 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.831592    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23ff7c29-eab4-46c8-831f-8aa51b684134-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.834982 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.834963    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9b4j\" (UniqueName: \"kubernetes.io/projected/23ff7c29-eab4-46c8-831f-8aa51b684134-kube-api-access-m9b4j\") pod \"alertmanager-main-0\" (UID: \"23ff7c29-eab4-46c8-831f-8aa51b684134\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:05.891985 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:05.891963    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:58:06.403484 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.403449    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"]
Jul 07 14:58:06.408828 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.408030    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.411729 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.411705    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 07 14:58:06.411860 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.411738    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 07 14:58:06.412085 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.412055    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f0o8b96q5jlod\""
Jul 07 14:58:06.412085 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.412075    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 07 14:58:06.412252 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.412068    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lnplm\""
Jul 07 14:58:06.412252 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.412126    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 07 14:58:06.412378 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.412361    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 07 14:58:06.418727 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.418705    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"]
Jul 07 14:58:06.530936 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.530910    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.531092 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.530948    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.531092 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.530981    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.531092 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.531046    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.531260 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.531111    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-grpc-tls\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.531260 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.531158    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e04a0fc-df5e-4d89-9a4b-e252354230a4-metrics-client-ca\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.531260 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.531192    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6rv2\" (UniqueName: \"kubernetes.io/projected/3e04a0fc-df5e-4d89-9a4b-e252354230a4-kube-api-access-g6rv2\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.531260 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.531216    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-tls\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.631954 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.631922    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-grpc-tls\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632103 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.631961    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e04a0fc-df5e-4d89-9a4b-e252354230a4-metrics-client-ca\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632103 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.631979    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6rv2\" (UniqueName: \"kubernetes.io/projected/3e04a0fc-df5e-4d89-9a4b-e252354230a4-kube-api-access-g6rv2\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632103 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.632002    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-tls\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632103 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.632039    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632103 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.632070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632348 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.632115    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632348 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.632155    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.632820 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.632758    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e04a0fc-df5e-4d89-9a4b-e252354230a4-metrics-client-ca\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.635341 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.635316    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.635503 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.635349    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.635503 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.635422    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-grpc-tls\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.635503 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.635473    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.635686 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.635600    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-tls\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.636785 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.636761    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e04a0fc-df5e-4d89-9a4b-e252354230a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.641090 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.641072    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6rv2\" (UniqueName: \"kubernetes.io/projected/3e04a0fc-df5e-4d89-9a4b-e252354230a4-kube-api-access-g6rv2\") pod \"thanos-querier-566f8c8df6-kc4b2\" (UID: \"3e04a0fc-df5e-4d89-9a4b-e252354230a4\") " pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:06.719827 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:06.719776    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:08.746013 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.745980    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"]
Jul 07 14:58:08.749490 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.749456    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.753519 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.753499    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 07 14:58:08.753625 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.753581    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 14:58:08.753625 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.753600    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-hbmp6\""
Jul 07 14:58:08.753707 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.753694    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 07 14:58:08.753831 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.753806    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7tg51d9s8qm9k\""
Jul 07 14:58:08.753940 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.753874    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 07 14:58:08.759042 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.758965    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:58:08.759042 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.759010    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:58:08.760845 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.760825    2556 patch_prober.go:28] interesting pod/console-f75dffcf9-z7m8p container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.20:8443/health\": dial tcp 10.132.0.20:8443: connect: connection refused" start-of-body=
Jul 07 14:58:08.760958 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.760867    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-f75dffcf9-z7m8p" podUID="ec0c2758-cf78-4673-be66-fee1ba7ff1d5" containerName="console" probeResult="failure" output="Get \"https://10.132.0.20:8443/health\": dial tcp 10.132.0.20:8443: connect: connection refused"
Jul 07 14:58:08.763913 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.763894    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"]
Jul 07 14:58:08.854767 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.854733    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/707b7933-24bd-4d4c-b895-9530f2477cbd-metrics-server-audit-profiles\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.854923 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.854775    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skl87\" (UniqueName: \"kubernetes.io/projected/707b7933-24bd-4d4c-b895-9530f2477cbd-kube-api-access-skl87\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.854923 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.854836    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-secret-metrics-server-tls\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.854923 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.854918    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-client-ca-bundle\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.855087 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.854946    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707b7933-24bd-4d4c-b895-9530f2477cbd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.855176 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.855135    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/707b7933-24bd-4d4c-b895-9530f2477cbd-audit-log\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.855293 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.855182    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-secret-metrics-server-client-certs\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.956110 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.956079    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/707b7933-24bd-4d4c-b895-9530f2477cbd-metrics-server-audit-profiles\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.956255 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.956120    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skl87\" (UniqueName: \"kubernetes.io/projected/707b7933-24bd-4d4c-b895-9530f2477cbd-kube-api-access-skl87\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.956255 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.956151    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-secret-metrics-server-tls\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.956255 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.956193    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-client-ca-bundle\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.956255 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.956223    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707b7933-24bd-4d4c-b895-9530f2477cbd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.956516 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.956493    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/707b7933-24bd-4d4c-b895-9530f2477cbd-audit-log\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.956651 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.956540    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-secret-metrics-server-client-certs\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.957063 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.957039    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/707b7933-24bd-4d4c-b895-9530f2477cbd-audit-log\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.957267 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.957207    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707b7933-24bd-4d4c-b895-9530f2477cbd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.957267 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.957206    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/707b7933-24bd-4d4c-b895-9530f2477cbd-metrics-server-audit-profiles\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.959007 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.958987    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-secret-metrics-server-client-certs\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.959138 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.959118    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-client-ca-bundle\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.959203 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.959186    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/707b7933-24bd-4d4c-b895-9530f2477cbd-secret-metrics-server-tls\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:08.965365 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:08.965337    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skl87\" (UniqueName: \"kubernetes.io/projected/707b7933-24bd-4d4c-b895-9530f2477cbd-kube-api-access-skl87\") pod \"metrics-server-fdbf56d4b-zzlp8\" (UID: \"707b7933-24bd-4d4c-b895-9530f2477cbd\") " pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:09.061264 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.061206    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:09.210787 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.210748    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5888f9f647-q8kr7"]
Jul 07 14:58:09.289019 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.288988    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cf55649d4-92xck"]
Jul 07 14:58:09.292724 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.292691    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.318313 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.318255    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf55649d4-92xck"]
Jul 07 14:58:09.360721 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.360687    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-trusted-ca-bundle\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.360853 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.360729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-console-config\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.360853 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.360759    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-serving-cert\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.360853 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.360834    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-service-ca\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.361018 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.360949    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-oauth-serving-cert\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.361070 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.361032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qhl7\" (UniqueName: \"kubernetes.io/projected/37fbb372-18e2-40ce-a87d-af0da19a3530-kube-api-access-9qhl7\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.361127 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.361088    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-oauth-config\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.461896 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.461852    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qhl7\" (UniqueName: \"kubernetes.io/projected/37fbb372-18e2-40ce-a87d-af0da19a3530-kube-api-access-9qhl7\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.461896 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.461899    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-oauth-config\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.462116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.461931    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-trusted-ca-bundle\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.462116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.461958    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-console-config\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.462116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.461986    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-serving-cert\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.462116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.462043    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-service-ca\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.462307 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.462142    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-oauth-serving-cert\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.462950 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.462905    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-console-config\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.462950 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.462906    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-service-ca\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.463121 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.462909    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-oauth-serving-cert\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.463281 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.463253    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-trusted-ca-bundle\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.465213 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.465170    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-oauth-config\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.465319 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.465230    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-serving-cert\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.471222 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.471197    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qhl7\" (UniqueName: \"kubernetes.io/projected/37fbb372-18e2-40ce-a87d-af0da19a3530-kube-api-access-9qhl7\") pod \"console-5cf55649d4-92xck\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") " pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:09.604554 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:09.604467    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:13.225667 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:13.225638    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31f5eb6_f858_4da0_98d4_7bf4ccb48d7e.slice/crio-93796b2166b98fd9e2add09d8fd0ecbdd524e26e9888a249dc1910ecd71f38c4 WatchSource:0}: Error finding container 93796b2166b98fd9e2add09d8fd0ecbdd524e26e9888a249dc1910ecd71f38c4: Status 404 returned error can't find the container with id 93796b2166b98fd9e2add09d8fd0ecbdd524e26e9888a249dc1910ecd71f38c4
Jul 07 14:58:13.512429 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.512398    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"]
Jul 07 14:58:13.513452 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:13.513431    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e04a0fc_df5e_4d89_9a4b_e252354230a4.slice/crio-cae5cf3e048cc21ceb10775138cf94d3ed73452d8893dfb58c4ec17a5620d76a WatchSource:0}: Error finding container cae5cf3e048cc21ceb10775138cf94d3ed73452d8893dfb58c4ec17a5620d76a: Status 404 returned error can't find the container with id cae5cf3e048cc21ceb10775138cf94d3ed73452d8893dfb58c4ec17a5620d76a
Jul 07 14:58:13.654231 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.654142    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 14:58:13.657294 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.657243    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"]
Jul 07 14:58:13.659028 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:13.658996    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ff7c29_eab4_46c8_831f_8aa51b684134.slice/crio-cf1e9cc389890bf519f192fb65de2cd42b8f83549024859e106d9e4c38aca577 WatchSource:0}: Error finding container cf1e9cc389890bf519f192fb65de2cd42b8f83549024859e106d9e4c38aca577: Status 404 returned error can't find the container with id cf1e9cc389890bf519f192fb65de2cd42b8f83549024859e106d9e4c38aca577
Jul 07 14:58:13.659245 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:13.659212    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707b7933_24bd_4d4c_b895_9530f2477cbd.slice/crio-2c3a7021710621e1c4482a45d91a4a6c6d9ca3d952e91854095dbd51780066c6 WatchSource:0}: Error finding container 2c3a7021710621e1c4482a45d91a4a6c6d9ca3d952e91854095dbd51780066c6: Status 404 returned error can't find the container with id 2c3a7021710621e1c4482a45d91a4a6c6d9ca3d952e91854095dbd51780066c6
Jul 07 14:58:13.660484 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.660398    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf55649d4-92xck"]
Jul 07 14:58:13.664152 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:13.664094    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37fbb372_18e2_40ce_a87d_af0da19a3530.slice/crio-3b7df46c932904a1e1e66a76bb502f0d65e265a98a664bd648fa666517082f47 WatchSource:0}: Error finding container 3b7df46c932904a1e1e66a76bb502f0d65e265a98a664bd648fa666517082f47: Status 404 returned error can't find the container with id 3b7df46c932904a1e1e66a76bb502f0d65e265a98a664bd648fa666517082f47
Jul 07 14:58:13.727669 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.727631    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xjl76" event={"ID":"0718570a-e9fe-460b-a0c9-b895125c2d44","Type":"ContainerStarted","Data":"0e20cf6685ec651c27544ce10be58416098858afbfe980de085b80535aec4434"}
Jul 07 14:58:13.727824 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.727699    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xjl76" event={"ID":"0718570a-e9fe-460b-a0c9-b895125c2d44","Type":"ContainerStarted","Data":"60f921208365c8ffab74cc277a80559b12be02fe7d8936a8a67086dc9b4d7082"}
Jul 07 14:58:13.728004 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.727957    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xjl76"
Jul 07 14:58:13.730191 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.730164    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" event={"ID":"3e04a0fc-df5e-4d89-9a4b-e252354230a4","Type":"ContainerStarted","Data":"cae5cf3e048cc21ceb10775138cf94d3ed73452d8893dfb58c4ec17a5620d76a"}
Jul 07 14:58:13.732153 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.732127    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf55649d4-92xck" event={"ID":"37fbb372-18e2-40ce-a87d-af0da19a3530","Type":"ContainerStarted","Data":"3b7df46c932904a1e1e66a76bb502f0d65e265a98a664bd648fa666517082f47"}
Jul 07 14:58:13.734204 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.734178    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"cf1e9cc389890bf519f192fb65de2cd42b8f83549024859e106d9e4c38aca577"}
Jul 07 14:58:13.736176 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.736151    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8" event={"ID":"707b7933-24bd-4d4c-b895-9530f2477cbd","Type":"ContainerStarted","Data":"2c3a7021710621e1c4482a45d91a4a6c6d9ca3d952e91854095dbd51780066c6"}
Jul 07 14:58:13.738479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.738073    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-php5t" event={"ID":"b1c843ae-67f8-4d36-80cc-6fb63018755a","Type":"ContainerStarted","Data":"f1edf31d890d20b583ce13414bf88b2ecea13588b247a4c7b72f0bbc440b9ada"}
Jul 07 14:58:13.738479 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.738229    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-php5t"
Jul 07 14:58:13.739952 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.739924    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77t6k" event={"ID":"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e","Type":"ContainerStarted","Data":"93796b2166b98fd9e2add09d8fd0ecbdd524e26e9888a249dc1910ecd71f38c4"}
Jul 07 14:58:13.743633 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.743608    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mb4h" event={"ID":"71190e89-69d2-4d28-8825-f3de21d2de28","Type":"ContainerStarted","Data":"5fa36fcff5ca3f2a5cdd34bac2e7e19700b151149ea0c2dff3fea4fe026a1906"}
Jul 07 14:58:13.751721 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.751600    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-php5t"
Jul 07 14:58:13.772903 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.771255    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6mb4h" podStartSLOduration=33.074316881 podStartE2EDuration="50.771236558s" totalImagesPullingTime="17.696919677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:57:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:13.770557498 +0000 UTC m=+111.119040185" watchObservedRunningTime="2026-07-07 14:58:13.771236558 +0000 UTC m=+111.119719169"
Jul 07 14:58:13.773516 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.773235    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xjl76" podStartSLOduration=65.013183692 podStartE2EDuration="1m15.773221347s" totalImagesPullingTime="10.760037655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:13.750043398 +0000 UTC m=+111.098526042" watchObservedRunningTime="2026-07-07 14:58:13.773221347 +0000 UTC m=+111.121703979"
Jul 07 14:58:13.792583 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:13.790357    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-php5t" podStartSLOduration=1.8142382129999999 podStartE2EDuration="20.790342438s" totalImagesPullingTime="18.976104225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:13.789236473 +0000 UTC m=+111.137719128" watchObservedRunningTime="2026-07-07 14:58:13.790342438 +0000 UTC m=+111.138825069"
Jul 07 14:58:14.754324 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:14.754212    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf55649d4-92xck" event={"ID":"37fbb372-18e2-40ce-a87d-af0da19a3530","Type":"ContainerStarted","Data":"572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d"}
Jul 07 14:58:14.778807 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:14.778031    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cf55649d4-92xck" podStartSLOduration=5.7780088769999995 podStartE2EDuration="5.778008877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:14.776333702 +0000 UTC m=+112.124816332" watchObservedRunningTime="2026-07-07 14:58:14.778008877 +0000 UTC m=+112.126491509"
Jul 07 14:58:15.763216 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:15.762591    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"81b08b0f85eaf63b4c17dfd64c2295d4a4e7bb5068515e14ea0dcaf90f4e2a71"}
Jul 07 14:58:15.766028 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:15.765997    2556 generic.go:356] "Generic (PLEG): container finished" podID="e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e" containerID="3a04f87c634e76033b6923d4f8b3b8fd5c666145311739e5981f99a21efb29a2" exitCode=0
Jul 07 14:58:15.766152 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:15.766086    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77t6k" event={"ID":"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e","Type":"ContainerDied","Data":"3a04f87c634e76033b6923d4f8b3b8fd5c666145311739e5981f99a21efb29a2"}
Jul 07 14:58:16.773887 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:16.773846    2556 generic.go:356] "Generic (PLEG): container finished" podID="23ff7c29-eab4-46c8-831f-8aa51b684134" containerID="81b08b0f85eaf63b4c17dfd64c2295d4a4e7bb5068515e14ea0dcaf90f4e2a71" exitCode=0
Jul 07 14:58:16.774449 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:16.773928    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerDied","Data":"81b08b0f85eaf63b4c17dfd64c2295d4a4e7bb5068515e14ea0dcaf90f4e2a71"}
Jul 07 14:58:17.287123 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.287094    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78c556786f-mfrd7"]
Jul 07 14:58:17.293157 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.293100    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:58:17.783746 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.783693    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" event={"ID":"3e04a0fc-df5e-4d89-9a4b-e252354230a4","Type":"ContainerStarted","Data":"18d7928d3ba8ac3a41aed3b6cd886b8f27d9d15068693dbd612d9ea04acd006a"}
Jul 07 14:58:17.784236 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.783849    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" event={"ID":"3e04a0fc-df5e-4d89-9a4b-e252354230a4","Type":"ContainerStarted","Data":"d56200a779c18f7eadb3a97c196629ae8ac0782c051939c254b708a515d0b6f1"}
Jul 07 14:58:17.784236 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.783894    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" event={"ID":"3e04a0fc-df5e-4d89-9a4b-e252354230a4","Type":"ContainerStarted","Data":"2447180a6a54ec89491f246d2b45337b6d42f5ace9e54e441c8402193dae54f9"}
Jul 07 14:58:17.785556 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.785527    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8" event={"ID":"707b7933-24bd-4d4c-b895-9530f2477cbd","Type":"ContainerStarted","Data":"7f10ebded6a36892642fbb7fed903f05ada1aadb85e7598af048dfe3998629cf"}
Jul 07 14:58:17.788385 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.788356    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77t6k" event={"ID":"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e","Type":"ContainerStarted","Data":"8f9bd1b2db4204c9f123d131a34b5f347da378b2376195f3a357fdc3af9ae914"}
Jul 07 14:58:17.788503 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.788410    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77t6k" event={"ID":"e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e","Type":"ContainerStarted","Data":"cfb107d41288231ef2873f9d7d78f3ee9cbd6e9732802f4e7ab49f7bb7869a4f"}
Jul 07 14:58:17.809208 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:17.809153    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8" podStartSLOduration=6.45935694 podStartE2EDuration="9.809137404s" totalImagesPullingTime="3.349780464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:17.808636535 +0000 UTC m=+115.157119179" watchObservedRunningTime="2026-07-07 14:58:17.809137404 +0000 UTC m=+115.157620034"
Jul 07 14:58:18.765055 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:18.764989    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:58:18.770251 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:18.770200    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:58:18.790151 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:18.790099    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-77t6k" podStartSLOduration=12.975297906 podStartE2EDuration="14.790080983s" totalImagesPullingTime="1.679000168s" totalInitContainerRuntime="135.782909ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:17.834980303 +0000 UTC m=+115.183462971" watchObservedRunningTime="2026-07-07 14:58:18.790080983 +0000 UTC m=+116.138563616"
Jul 07 14:58:19.604594 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:19.604552    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:19.604778 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:19.604620    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:19.610241 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:19.610214    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:19.803269 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:19.803247    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 14:58:19.877928 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:19.877347    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f75dffcf9-z7m8p"]
Jul 07 14:58:20.806345 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.806299    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" event={"ID":"3e04a0fc-df5e-4d89-9a4b-e252354230a4","Type":"ContainerStarted","Data":"de1c3ced9858f021987ac476766b14b2ac7268d424d0012511dad0e271d2f3c8"}
Jul 07 14:58:20.806832 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.806428    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:20.806832 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.806471    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" event={"ID":"3e04a0fc-df5e-4d89-9a4b-e252354230a4","Type":"ContainerStarted","Data":"f12adf7c8c097e0747d0481df72056f2f07d190f196d2202ab872dad89839052"}
Jul 07 14:58:20.806832 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.806516    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" event={"ID":"3e04a0fc-df5e-4d89-9a4b-e252354230a4","Type":"ContainerStarted","Data":"68a9e272c1640cf33999743430f919f31c048680d57acf7bda53a91432ccca72"}
Jul 07 14:58:20.810211 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.810180    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"b3b721a2358a91a833a89c424e4b1885c67b790056904222e9e699f18f6b8afe"}
Jul 07 14:58:20.810424 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.810250    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"669190092c9c4e597a5702c83e9dcadeeb0dcc34e48c21d7dd6a005bec63a655"}
Jul 07 14:58:20.810424 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.810291    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"7387182b7b6ab210888e915ff6a40b61043d2734433c72ea3ac6d1ed42a9d07f"}
Jul 07 14:58:20.810424 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.810333    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"1397bdca5e400e619330deeb46afc6de0457b2bf052fb96955bb4bfe1fb9e42d"}
Jul 07 14:58:20.810424 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.810366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"6ecb084214ac1fecc6d259206ab1556bed6753bbf1e9ef06897bde9ee9ab4f16"}
Jul 07 14:58:20.810424 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.810401    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23ff7c29-eab4-46c8-831f-8aa51b684134","Type":"ContainerStarted","Data":"f4a865c9b5b6d4384145697daab2a850af60963177416925034a6adfe29fdfec"}
Jul 07 14:58:20.836168 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.836116    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2" podStartSLOduration=9.085020168 podStartE2EDuration="14.836097257s" totalImagesPullingTime="5.751077089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:20.833743983 +0000 UTC m=+118.182226615" watchObservedRunningTime="2026-07-07 14:58:20.836097257 +0000 UTC m=+118.184579888"
Jul 07 14:58:20.861539 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:20.861497    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.368066239000001 podStartE2EDuration="15.86148354s" totalImagesPullingTime="4.477090009s" totalInitContainerRuntime="16.327292ms" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:20.86141494 +0000 UTC m=+118.209897592" watchObservedRunningTime="2026-07-07 14:58:20.86148354 +0000 UTC m=+118.209966170"
Jul 07 14:58:23.758332 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:23.758299    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xjl76"
Jul 07 14:58:26.822137 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:26.822107    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-566f8c8df6-kc4b2"
Jul 07 14:58:29.061732 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:29.061691    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:29.062208 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:29.061757    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:32.070163 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:32.070123    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:58:32.072554 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:32.072529    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6fd483f-453e-4289-82fb-02ad4d241017-metrics-certs\") pod \"network-metrics-daemon-wxd9k\" (UID: \"b6fd483f-453e-4289-82fb-02ad4d241017\") " pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:58:32.294043 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:32.294013    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8fcz7\""
Jul 07 14:58:32.301462 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:32.301443    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxd9k"
Jul 07 14:58:32.505576 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:32.505539    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxd9k"]
Jul 07 14:58:32.509664 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:58:32.509638    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6fd483f_453e_4289_82fb_02ad4d241017.slice/crio-5a158ea07ea1d5cd2161663943e3fecf20eb051ce0fb1759d1d22a863d32f416 WatchSource:0}: Error finding container 5a158ea07ea1d5cd2161663943e3fecf20eb051ce0fb1759d1d22a863d32f416: Status 404 returned error can't find the container with id 5a158ea07ea1d5cd2161663943e3fecf20eb051ce0fb1759d1d22a863d32f416
Jul 07 14:58:32.861995 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:32.861966    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxd9k" event={"ID":"b6fd483f-453e-4289-82fb-02ad4d241017","Type":"ContainerStarted","Data":"5a158ea07ea1d5cd2161663943e3fecf20eb051ce0fb1759d1d22a863d32f416"}
Jul 07 14:58:34.240369 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.240317    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5888f9f647-q8kr7" podUID="4bdff553-4c5c-40be-ab27-d98f6af894ee" containerName="console" containerID="cri-o://489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff" gracePeriod=15
Jul 07 14:58:34.525271 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.525240    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:58:34.591499 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.591469    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-serving-cert\") pod \"4bdff553-4c5c-40be-ab27-d98f6af894ee\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") "
Jul 07 14:58:34.591635 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.591512    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6tv7\" (UniqueName: \"kubernetes.io/projected/4bdff553-4c5c-40be-ab27-d98f6af894ee-kube-api-access-x6tv7\") pod \"4bdff553-4c5c-40be-ab27-d98f6af894ee\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") "
Jul 07 14:58:34.591635 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.591545    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-config\") pod \"4bdff553-4c5c-40be-ab27-d98f6af894ee\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") "
Jul 07 14:58:34.591635 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.591575    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-oauth-serving-cert\") pod \"4bdff553-4c5c-40be-ab27-d98f6af894ee\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") "
Jul 07 14:58:34.591784 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.591662    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-service-ca\") pod \"4bdff553-4c5c-40be-ab27-d98f6af894ee\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") "
Jul 07 14:58:34.591784 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.591701    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-oauth-config\") pod \"4bdff553-4c5c-40be-ab27-d98f6af894ee\" (UID: \"4bdff553-4c5c-40be-ab27-d98f6af894ee\") "
Jul 07 14:58:34.592002 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.591974    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-config" (OuterVolumeSpecName: "console-config") pod "4bdff553-4c5c-40be-ab27-d98f6af894ee" (UID: "4bdff553-4c5c-40be-ab27-d98f6af894ee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:34.592086 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.592060    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4bdff553-4c5c-40be-ab27-d98f6af894ee" (UID: "4bdff553-4c5c-40be-ab27-d98f6af894ee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:34.592154 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.592064    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-service-ca" (OuterVolumeSpecName: "service-ca") pod "4bdff553-4c5c-40be-ab27-d98f6af894ee" (UID: "4bdff553-4c5c-40be-ab27-d98f6af894ee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:34.593860 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.593786    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4bdff553-4c5c-40be-ab27-d98f6af894ee" (UID: "4bdff553-4c5c-40be-ab27-d98f6af894ee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:34.594106 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.594087    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4bdff553-4c5c-40be-ab27-d98f6af894ee" (UID: "4bdff553-4c5c-40be-ab27-d98f6af894ee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:34.594148 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.594112    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bdff553-4c5c-40be-ab27-d98f6af894ee-kube-api-access-x6tv7" (OuterVolumeSpecName: "kube-api-access-x6tv7") pod "4bdff553-4c5c-40be-ab27-d98f6af894ee" (UID: "4bdff553-4c5c-40be-ab27-d98f6af894ee"). InnerVolumeSpecName "kube-api-access-x6tv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:34.692584 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.692561    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-oauth-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:34.692584 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.692581    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-service-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:34.692711 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.692590    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-oauth-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:34.692711 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.692599    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:34.692711 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.692609    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6tv7\" (UniqueName: \"kubernetes.io/projected/4bdff553-4c5c-40be-ab27-d98f6af894ee-kube-api-access-x6tv7\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:34.692711 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.692617    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bdff553-4c5c-40be-ab27-d98f6af894ee-console-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:34.871268 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.871186    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxd9k" event={"ID":"b6fd483f-453e-4289-82fb-02ad4d241017","Type":"ContainerStarted","Data":"ded5331cbc956f613533ccf848c9327911101669472a52a3a253e3b609ea3a0a"}
Jul 07 14:58:34.871408 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.871277    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxd9k" event={"ID":"b6fd483f-453e-4289-82fb-02ad4d241017","Type":"ContainerStarted","Data":"3e2d8e6bc6e2a03d51ab0f0ba4bcf5ffba89d253b9c2b5dd22b722e87afd0bc0"}
Jul 07 14:58:34.872326 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.872300    2556 generic.go:356] "Generic (PLEG): container finished" podID="4bdff553-4c5c-40be-ab27-d98f6af894ee" containerID="489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff" exitCode=0
Jul 07 14:58:34.872430 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.872336    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5888f9f647-q8kr7"
Jul 07 14:58:34.872486 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.872339    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5888f9f647-q8kr7" event={"ID":"4bdff553-4c5c-40be-ab27-d98f6af894ee","Type":"ContainerDied","Data":"489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff"}
Jul 07 14:58:34.872486 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.872460    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5888f9f647-q8kr7" event={"ID":"4bdff553-4c5c-40be-ab27-d98f6af894ee","Type":"ContainerDied","Data":"5107f79a751f1f400a0fcf64445778c46650762d6981803e1a88000ac6242f53"}
Jul 07 14:58:34.872486 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.872481    2556 scope.go:117] "RemoveContainer" containerID="489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff"
Jul 07 14:58:34.881001 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.880986    2556 scope.go:117] "RemoveContainer" containerID="489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff"
Jul 07 14:58:34.881232 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:58:34.881213    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff\": container with ID starting with 489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff not found: ID does not exist" containerID="489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff"
Jul 07 14:58:34.881314 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.881239    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff"} err="failed to get container status \"489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff\": rpc error: code = NotFound desc = could not find container \"489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff\": container with ID starting with 489f80ae8154ec42e6b0b58fe73c96e9a91e6f59443f8a8baf91b024e7e063ff not found: ID does not exist"
Jul 07 14:58:34.911435 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.911393    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wxd9k" podStartSLOduration=129.483699963 podStartE2EDuration="2m10.911381296s" totalImagesPullingTime="1.427681333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:34.910199865 +0000 UTC m=+132.258682494" watchObservedRunningTime="2026-07-07 14:58:34.911381296 +0000 UTC m=+132.259863925"
Jul 07 14:58:34.944111 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.944088    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5888f9f647-q8kr7"]
Jul 07 14:58:34.962712 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:34.962692    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5888f9f647-q8kr7"]
Jul 07 14:58:35.269484 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:35.269452    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bdff553-4c5c-40be-ab27-d98f6af894ee" path="/var/lib/kubelet/pods/4bdff553-4c5c-40be-ab27-d98f6af894ee/volumes"
Jul 07 14:58:42.320528 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.320457    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78c556786f-mfrd7" podUID="012cad90-75d4-490c-82fe-fba736164700" containerName="registry" containerID="cri-o://b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077" gracePeriod=30
Jul 07 14:58:42.563529 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.563508    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:58:42.655263 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655204    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-bound-sa-token\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.655263 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655251    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-trusted-ca\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.655414 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655276    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-registry-certificates\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.655414 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655291    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.655414 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655313    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-installation-pull-secrets\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.655414 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655337    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2vld\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-kube-api-access-d2vld\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.655591 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655424    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/012cad90-75d4-490c-82fe-fba736164700-ca-trust-extracted\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.655591 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655478    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-image-registry-private-configuration\") pod \"012cad90-75d4-490c-82fe-fba736164700\" (UID: \"012cad90-75d4-490c-82fe-fba736164700\") "
Jul 07 14:58:42.656057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.655854    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:42.656057 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.656014    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:42.657753 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.657726    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-kube-api-access-d2vld" (OuterVolumeSpecName: "kube-api-access-d2vld") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "kube-api-access-d2vld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:42.657855 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.657748    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:42.657981 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.657956    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:42.658059 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.657980    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:42.658116 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.658095    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:42.664306 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.664284    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012cad90-75d4-490c-82fe-fba736164700-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "012cad90-75d4-490c-82fe-fba736164700" (UID: "012cad90-75d4-490c-82fe-fba736164700"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:58:42.756090 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756067    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-image-registry-private-configuration\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.756190 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756091    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-bound-sa-token\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.756190 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756105    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-trusted-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.756190 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756118    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/012cad90-75d4-490c-82fe-fba736164700-registry-certificates\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.756190 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756130    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-registry-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.756190 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756142    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/012cad90-75d4-490c-82fe-fba736164700-installation-pull-secrets\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.756190 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756154    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2vld\" (UniqueName: \"kubernetes.io/projected/012cad90-75d4-490c-82fe-fba736164700-kube-api-access-d2vld\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.756190 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.756169    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/012cad90-75d4-490c-82fe-fba736164700-ca-trust-extracted\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:42.906677 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.906614    2556 generic.go:356] "Generic (PLEG): container finished" podID="012cad90-75d4-490c-82fe-fba736164700" containerID="b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077" exitCode=0
Jul 07 14:58:42.906677 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.906662    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c556786f-mfrd7"
Jul 07 14:58:42.906863 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.906696    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c556786f-mfrd7" event={"ID":"012cad90-75d4-490c-82fe-fba736164700","Type":"ContainerDied","Data":"b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077"}
Jul 07 14:58:42.906863 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.906742    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c556786f-mfrd7" event={"ID":"012cad90-75d4-490c-82fe-fba736164700","Type":"ContainerDied","Data":"3cbf9ad745bbb79bb0efbd982d7a1f41862629dd9cd2e671cacb077e7aaad7a5"}
Jul 07 14:58:42.906863 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.906762    2556 scope.go:117] "RemoveContainer" containerID="b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077"
Jul 07 14:58:42.915535 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.915520    2556 scope.go:117] "RemoveContainer" containerID="b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077"
Jul 07 14:58:42.915772 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:58:42.915753    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077\": container with ID starting with b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077 not found: ID does not exist" containerID="b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077"
Jul 07 14:58:42.915839 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.915780    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077"} err="failed to get container status \"b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077\": rpc error: code = NotFound desc = could not find container \"b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077\": container with ID starting with b0980b832b1805277cd873c62dff3f984cddefebe6bfa398fc2c341112567077 not found: ID does not exist"
Jul 07 14:58:42.928529 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.928509    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78c556786f-mfrd7"]
Jul 07 14:58:42.931921 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:42.931902    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78c556786f-mfrd7"]
Jul 07 14:58:43.273488 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:43.273460    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="012cad90-75d4-490c-82fe-fba736164700" path="/var/lib/kubelet/pods/012cad90-75d4-490c-82fe-fba736164700/volumes"
Jul 07 14:58:44.907970 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:44.907896    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f75dffcf9-z7m8p" podUID="ec0c2758-cf78-4673-be66-fee1ba7ff1d5" containerName="console" containerID="cri-o://8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4" gracePeriod=15
Jul 07 14:58:45.182293 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.182268    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:58:45.273975 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.273946    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-oauth-config\") pod \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") "
Jul 07 14:58:45.274112 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.273996    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-oauth-serving-cert\") pod \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") "
Jul 07 14:58:45.274112 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274018    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-service-ca\") pod \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") "
Jul 07 14:58:45.274112 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274042    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-trusted-ca-bundle\") pod \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") "
Jul 07 14:58:45.274112 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274066    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwrv6\" (UniqueName: \"kubernetes.io/projected/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-kube-api-access-zwrv6\") pod \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") "
Jul 07 14:58:45.274311 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274120    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-config\") pod \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") "
Jul 07 14:58:45.274311 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274234    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-serving-cert\") pod \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\" (UID: \"ec0c2758-cf78-4673-be66-fee1ba7ff1d5\") "
Jul 07 14:58:45.274538 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274506    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec0c2758-cf78-4673-be66-fee1ba7ff1d5" (UID: "ec0c2758-cf78-4673-be66-fee1ba7ff1d5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:45.274538 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274526    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec0c2758-cf78-4673-be66-fee1ba7ff1d5" (UID: "ec0c2758-cf78-4673-be66-fee1ba7ff1d5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:45.274720 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274532    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ec0c2758-cf78-4673-be66-fee1ba7ff1d5" (UID: "ec0c2758-cf78-4673-be66-fee1ba7ff1d5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:45.274720 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.274574    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-config" (OuterVolumeSpecName: "console-config") pod "ec0c2758-cf78-4673-be66-fee1ba7ff1d5" (UID: "ec0c2758-cf78-4673-be66-fee1ba7ff1d5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:45.276041 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.276017    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec0c2758-cf78-4673-be66-fee1ba7ff1d5" (UID: "ec0c2758-cf78-4673-be66-fee1ba7ff1d5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:45.276436 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.276416    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec0c2758-cf78-4673-be66-fee1ba7ff1d5" (UID: "ec0c2758-cf78-4673-be66-fee1ba7ff1d5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:45.276508 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.276474    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-kube-api-access-zwrv6" (OuterVolumeSpecName: "kube-api-access-zwrv6") pod "ec0c2758-cf78-4673-be66-fee1ba7ff1d5" (UID: "ec0c2758-cf78-4673-be66-fee1ba7ff1d5"). InnerVolumeSpecName "kube-api-access-zwrv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:45.374905 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.374875    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-oauth-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:45.374905 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.374897    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-service-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:45.374905 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.374907    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-trusted-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:45.375075 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.374915    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwrv6\" (UniqueName: \"kubernetes.io/projected/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-kube-api-access-zwrv6\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:45.375075 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.374923    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:45.375075 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.374932    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:45.375075 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.374939    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0c2758-cf78-4673-be66-fee1ba7ff1d5-console-oauth-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 14:58:45.920758 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.920727    2556 generic.go:356] "Generic (PLEG): container finished" podID="ec0c2758-cf78-4673-be66-fee1ba7ff1d5" containerID="8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4" exitCode=0
Jul 07 14:58:45.921134 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.920771    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75dffcf9-z7m8p"
Jul 07 14:58:45.921134 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.920771    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75dffcf9-z7m8p" event={"ID":"ec0c2758-cf78-4673-be66-fee1ba7ff1d5","Type":"ContainerDied","Data":"8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4"}
Jul 07 14:58:45.921134 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.920929    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75dffcf9-z7m8p" event={"ID":"ec0c2758-cf78-4673-be66-fee1ba7ff1d5","Type":"ContainerDied","Data":"4d58c4f4efae0f5f870a9a5f5635a0808d6380f7f44bf53719b9d66a05b5dfab"}
Jul 07 14:58:45.921134 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.920965    2556 scope.go:117] "RemoveContainer" containerID="8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4"
Jul 07 14:58:45.929752 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.929736    2556 scope.go:117] "RemoveContainer" containerID="8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4"
Jul 07 14:58:45.929997 ip-10-0-134-178 kubenswrapper[2556]: E0707 14:58:45.929979    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4\": container with ID starting with 8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4 not found: ID does not exist" containerID="8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4"
Jul 07 14:58:45.930053 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.930005    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4"} err="failed to get container status \"8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4\": rpc error: code = NotFound desc = could not find container \"8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4\": container with ID starting with 8aefe820404155db801111e8c1c13c6f9a68473d003628a349470c36bde32cb4 not found: ID does not exist"
Jul 07 14:58:45.943674 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.943653    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f75dffcf9-z7m8p"]
Jul 07 14:58:45.947389 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:45.947372    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f75dffcf9-z7m8p"]
Jul 07 14:58:47.269406 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:47.269370    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec0c2758-cf78-4673-be66-fee1ba7ff1d5" path="/var/lib/kubelet/pods/ec0c2758-cf78-4673-be66-fee1ba7ff1d5/volumes"
Jul 07 14:58:49.066973 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:49.066948    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:49.070643 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:49.070620    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-fdbf56d4b-zzlp8"
Jul 07 14:58:52.949236 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:52.949201    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2992c03-9dce-432d-a63e-99a5c3723d06" containerID="a06bb1cfba65358f47d33ceb739e6a81d816fa12b879a0b9c06decee2e112d5e" exitCode=0
Jul 07 14:58:52.949710 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:52.949285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf" event={"ID":"c2992c03-9dce-432d-a63e-99a5c3723d06","Type":"ContainerDied","Data":"a06bb1cfba65358f47d33ceb739e6a81d816fa12b879a0b9c06decee2e112d5e"}
Jul 07 14:58:52.949710 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:52.949702    2556 scope.go:117] "RemoveContainer" containerID="a06bb1cfba65358f47d33ceb739e6a81d816fa12b879a0b9c06decee2e112d5e"
Jul 07 14:58:53.955244 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:58:53.955214    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-n5ptf" event={"ID":"c2992c03-9dce-432d-a63e-99a5c3723d06","Type":"ContainerStarted","Data":"99e2337154df7e11b4f017e69a5b993c38a1a9dd612dfb37e404c2ff8fced032"}
Jul 07 14:59:28.922851 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.922815    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"]
Jul 07 14:59:28.923251 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923148    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012cad90-75d4-490c-82fe-fba736164700" containerName="registry"
Jul 07 14:59:28.923251 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923163    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="012cad90-75d4-490c-82fe-fba736164700" containerName="registry"
Jul 07 14:59:28.923251 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923180    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec0c2758-cf78-4673-be66-fee1ba7ff1d5" containerName="console"
Jul 07 14:59:28.923251 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923185    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec0c2758-cf78-4673-be66-fee1ba7ff1d5" containerName="console"
Jul 07 14:59:28.923251 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923198    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bdff553-4c5c-40be-ab27-d98f6af894ee" containerName="console"
Jul 07 14:59:28.923251 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923204    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bdff553-4c5c-40be-ab27-d98f6af894ee" containerName="console"
Jul 07 14:59:28.923437 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923263    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec0c2758-cf78-4673-be66-fee1ba7ff1d5" containerName="console"
Jul 07 14:59:28.923437 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923271    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bdff553-4c5c-40be-ab27-d98f6af894ee" containerName="console"
Jul 07 14:59:28.923437 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.923279    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="012cad90-75d4-490c-82fe-fba736164700" containerName="registry"
Jul 07 14:59:28.926701 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.926679    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:28.929321 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.929283    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6dj28\""
Jul 07 14:59:28.929425 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.929320    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 07 14:59:28.929425 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.929361    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 07 14:59:28.929903 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.929886    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 07 14:59:28.930544 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.930529    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 07 14:59:28.931027 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.931012    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 07 14:59:28.936724 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.936705    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 07 14:59:28.946052 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:28.946031    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"]
Jul 07 14:59:29.107555 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107530    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-telemeter-client-tls\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.107663 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107557    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-secret-telemeter-client\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.107663 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107586    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-metrics-client-ca\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.107663 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107626    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-federate-client-tls\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.107762 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107661    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.107762 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107689    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.107762 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107728    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp755\" (UniqueName: \"kubernetes.io/projected/059c013a-575c-4397-9d5b-90770213f152-kube-api-access-gp755\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.107902 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.107766    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-serving-certs-ca-bundle\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208123 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-metrics-client-ca\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208123 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208096    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-federate-client-tls\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208123 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208117    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208275 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208138    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208275 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208179    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp755\" (UniqueName: \"kubernetes.io/projected/059c013a-575c-4397-9d5b-90770213f152-kube-api-access-gp755\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208275 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208220    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-serving-certs-ca-bundle\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208275 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-telemeter-client-tls\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.208452 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.208285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-secret-telemeter-client\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.209121 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.209094    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-serving-certs-ca-bundle\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.209121 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.209125    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-metrics-client-ca\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.209335 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.209189    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/059c013a-575c-4397-9d5b-90770213f152-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.210975 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.210944    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.211069 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.211048    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-federate-client-tls\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.211280 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.211256    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-telemeter-client-tls\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.211318 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.211302    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/059c013a-575c-4397-9d5b-90770213f152-secret-telemeter-client\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.218469 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.218443    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp755\" (UniqueName: \"kubernetes.io/projected/059c013a-575c-4397-9d5b-90770213f152-kube-api-access-gp755\") pod \"telemeter-client-5c7f6c5c9b-8b5xg\" (UID: \"059c013a-575c-4397-9d5b-90770213f152\") " pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.236379 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.236355    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"
Jul 07 14:59:29.403714 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:29.403676    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg"]
Jul 07 14:59:29.404857 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:59:29.404829    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059c013a_575c_4397_9d5b_90770213f152.slice/crio-8df969a925ee7e3dcbffedb3dce89859370b94dc11480e691e1f441cb1515a0d WatchSource:0}: Error finding container 8df969a925ee7e3dcbffedb3dce89859370b94dc11480e691e1f441cb1515a0d: Status 404 returned error can't find the container with id 8df969a925ee7e3dcbffedb3dce89859370b94dc11480e691e1f441cb1515a0d
Jul 07 14:59:30.110938 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:30.110897    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg" event={"ID":"059c013a-575c-4397-9d5b-90770213f152","Type":"ContainerStarted","Data":"8df969a925ee7e3dcbffedb3dce89859370b94dc11480e691e1f441cb1515a0d"}
Jul 07 14:59:32.122465 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.122420    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg" event={"ID":"059c013a-575c-4397-9d5b-90770213f152","Type":"ContainerStarted","Data":"ec56a088c6f8098a3be6b05c2008c44549979eaa40aaac7f34fbda6c51dba2e6"}
Jul 07 14:59:32.122882 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.122498    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg" event={"ID":"059c013a-575c-4397-9d5b-90770213f152","Type":"ContainerStarted","Data":"859013112c9125615cad94f104cd08234c8eaac977684d4c01b87d491a45eb03"}
Jul 07 14:59:32.122882 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.122531    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg" event={"ID":"059c013a-575c-4397-9d5b-90770213f152","Type":"ContainerStarted","Data":"ecd5c121331f475dd050cff2bfe0147610e9e443b26e470d512f9d2e6bfd02f0"}
Jul 07 14:59:32.147657 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.147589    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5c7f6c5c9b-8b5xg" podStartSLOduration=2.206620224 podStartE2EDuration="4.147567181s" totalImagesPullingTime="1.940946957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:32.145334185 +0000 UTC m=+189.493816812" watchObservedRunningTime="2026-07-07 14:59:32.147567181 +0000 UTC m=+189.496049813"
Jul 07 14:59:32.711345 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.711312    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d96ccc7cb-txh5r"]
Jul 07 14:59:32.713633 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.713610    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.725746 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.725721    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d96ccc7cb-txh5r"]
Jul 07 14:59:32.833788 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.833766    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-console-config\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.833939 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.833821    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfk7z\" (UniqueName: \"kubernetes.io/projected/5a63e8b7-4368-40e5-99e7-478534b5a036-kube-api-access-lfk7z\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.833939 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.833841    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-service-ca\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.833939 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.833862    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-serving-cert\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.833939 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.833922    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-oauth-config\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.834089 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.833955    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-trusted-ca-bundle\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.834089 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.833974    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-oauth-serving-cert\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.934916 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.934891    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-serving-cert\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935036 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.934935    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-oauth-config\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935036 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.934950    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-trusted-ca-bundle\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935036 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.934967    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-oauth-serving-cert\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935183 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.935149    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-console-config\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.935189    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfk7z\" (UniqueName: \"kubernetes.io/projected/5a63e8b7-4368-40e5-99e7-478534b5a036-kube-api-access-lfk7z\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.935217    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-service-ca\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935741 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.935718    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-oauth-serving-cert\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935851 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.935823    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-service-ca\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935932 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.935911    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-trusted-ca-bundle\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.935993 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.935911    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-console-config\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.937355 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.937336    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-serving-cert\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.937434 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.937361    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-oauth-config\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:32.943488 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:32.943468    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfk7z\" (UniqueName: \"kubernetes.io/projected/5a63e8b7-4368-40e5-99e7-478534b5a036-kube-api-access-lfk7z\") pod \"console-5d96ccc7cb-txh5r\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") " pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:33.024091 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:33.024039    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:33.158247 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:33.158224    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d96ccc7cb-txh5r"]
Jul 07 14:59:33.160651 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:59:33.160624    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a63e8b7_4368_40e5_99e7_478534b5a036.slice/crio-a66310e58541e2a3dc21598075c1940f1e41c3cd34b328404ee3991f901b0fcb WatchSource:0}: Error finding container a66310e58541e2a3dc21598075c1940f1e41c3cd34b328404ee3991f901b0fcb: Status 404 returned error can't find the container with id a66310e58541e2a3dc21598075c1940f1e41c3cd34b328404ee3991f901b0fcb
Jul 07 14:59:34.132761 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:34.132715    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d96ccc7cb-txh5r" event={"ID":"5a63e8b7-4368-40e5-99e7-478534b5a036","Type":"ContainerStarted","Data":"3486c1347dc2c9e331f36249aded63112c7ab9ed6ac290ce2533cc721b7ebfec"}
Jul 07 14:59:34.132761 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:34.132770    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d96ccc7cb-txh5r" event={"ID":"5a63e8b7-4368-40e5-99e7-478534b5a036","Type":"ContainerStarted","Data":"a66310e58541e2a3dc21598075c1940f1e41c3cd34b328404ee3991f901b0fcb"}
Jul 07 14:59:34.154392 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:34.154337    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d96ccc7cb-txh5r" podStartSLOduration=2.154291292 podStartE2EDuration="2.154291292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:34.15339516 +0000 UTC m=+191.501877816" watchObservedRunningTime="2026-07-07 14:59:34.154291292 +0000 UTC m=+191.502773921"
Jul 07 14:59:42.088571 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.088540    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d96ccc7cb-txh5r"]
Jul 07 14:59:42.120718 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.120694    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f49b9ccbf-nc5tf"]
Jul 07 14:59:42.123096 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.123069    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.135008 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.134986    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f49b9ccbf-nc5tf"]
Jul 07 14:59:42.298657 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.298624    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-config\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.298824 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.298664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-serving-cert\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.298824 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.298696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckxn4\" (UniqueName: \"kubernetes.io/projected/f69eedeb-977f-4072-bd6e-91cbf308f9c8-kube-api-access-ckxn4\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.298824 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.298779    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-oauth-config\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.298984 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.298843    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-oauth-serving-cert\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.298984 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.298885    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-service-ca\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.298984 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.298955    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-trusted-ca-bundle\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400072 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400011    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-config\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400072 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400046    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-serving-cert\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckxn4\" (UniqueName: \"kubernetes.io/projected/f69eedeb-977f-4072-bd6e-91cbf308f9c8-kube-api-access-ckxn4\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400098    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-oauth-config\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-oauth-serving-cert\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400154    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-service-ca\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400239 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-trusted-ca-bundle\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.400890 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400862    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-config\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.401017 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400909    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-service-ca\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.401017 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.400971    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-oauth-serving-cert\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.401370 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.401348    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f69eedeb-977f-4072-bd6e-91cbf308f9c8-trusted-ca-bundle\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.402685 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.402654    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-serving-cert\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.402809 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.402777    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f69eedeb-977f-4072-bd6e-91cbf308f9c8-console-oauth-config\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.412221 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.412202    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckxn4\" (UniqueName: \"kubernetes.io/projected/f69eedeb-977f-4072-bd6e-91cbf308f9c8-kube-api-access-ckxn4\") pod \"console-f49b9ccbf-nc5tf\" (UID: \"f69eedeb-977f-4072-bd6e-91cbf308f9c8\") " pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.432832 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.432809    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:42.558841 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:42.558822    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f49b9ccbf-nc5tf"]
Jul 07 14:59:42.561098 ip-10-0-134-178 kubenswrapper[2556]: W0707 14:59:42.561068    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69eedeb_977f_4072_bd6e_91cbf308f9c8.slice/crio-dfa0aae07ed050e0f2200971c00678bb7dc5947a24991a892c3c09ab04d8cea1 WatchSource:0}: Error finding container dfa0aae07ed050e0f2200971c00678bb7dc5947a24991a892c3c09ab04d8cea1: Status 404 returned error can't find the container with id dfa0aae07ed050e0f2200971c00678bb7dc5947a24991a892c3c09ab04d8cea1
Jul 07 14:59:43.024604 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:43.024572    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 14:59:43.173127 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:43.173094    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f49b9ccbf-nc5tf" event={"ID":"f69eedeb-977f-4072-bd6e-91cbf308f9c8","Type":"ContainerStarted","Data":"67e8dadff35a0f88dbecbc477478f1bf8699d175703b97f83f134c8c9aad65b7"}
Jul 07 14:59:43.173515 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:43.173145    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f49b9ccbf-nc5tf" event={"ID":"f69eedeb-977f-4072-bd6e-91cbf308f9c8","Type":"ContainerStarted","Data":"dfa0aae07ed050e0f2200971c00678bb7dc5947a24991a892c3c09ab04d8cea1"}
Jul 07 14:59:43.194716 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:43.194666    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f49b9ccbf-nc5tf" podStartSLOduration=1.19465098 podStartE2EDuration="1.19465098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:43.193445515 +0000 UTC m=+200.541928147" watchObservedRunningTime="2026-07-07 14:59:43.19465098 +0000 UTC m=+200.543133610"
Jul 07 14:59:52.433514 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:52.433472    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:52.433956 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:52.433540    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:52.438291 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:52.438263    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:53.221500 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:53.221471    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f49b9ccbf-nc5tf"
Jul 07 14:59:53.275107 ip-10-0-134-178 kubenswrapper[2556]: I0707 14:59:53.275079    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf55649d4-92xck"]
Jul 07 15:00:07.108159 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.108109    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5d96ccc7cb-txh5r" podUID="5a63e8b7-4368-40e5-99e7-478534b5a036" containerName="console" containerID="cri-o://3486c1347dc2c9e331f36249aded63112c7ab9ed6ac290ce2533cc721b7ebfec" gracePeriod=15
Jul 07 15:00:07.278586 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.278555    2556 generic.go:356] "Generic (PLEG): container finished" podID="5a63e8b7-4368-40e5-99e7-478534b5a036" containerID="3486c1347dc2c9e331f36249aded63112c7ab9ed6ac290ce2533cc721b7ebfec" exitCode=0
Jul 07 15:00:07.278704 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.278622    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d96ccc7cb-txh5r" event={"ID":"5a63e8b7-4368-40e5-99e7-478534b5a036","Type":"ContainerDied","Data":"3486c1347dc2c9e331f36249aded63112c7ab9ed6ac290ce2533cc721b7ebfec"}
Jul 07 15:00:07.355486 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.355463    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 15:00:07.387440 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387383    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfk7z\" (UniqueName: \"kubernetes.io/projected/5a63e8b7-4368-40e5-99e7-478534b5a036-kube-api-access-lfk7z\") pod \"5a63e8b7-4368-40e5-99e7-478534b5a036\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") "
Jul 07 15:00:07.387440 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387430    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-service-ca\") pod \"5a63e8b7-4368-40e5-99e7-478534b5a036\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") "
Jul 07 15:00:07.387568 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387494    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-oauth-serving-cert\") pod \"5a63e8b7-4368-40e5-99e7-478534b5a036\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") "
Jul 07 15:00:07.387568 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387532    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-console-config\") pod \"5a63e8b7-4368-40e5-99e7-478534b5a036\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") "
Jul 07 15:00:07.387658 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387564    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-serving-cert\") pod \"5a63e8b7-4368-40e5-99e7-478534b5a036\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") "
Jul 07 15:00:07.387658 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387627    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-oauth-config\") pod \"5a63e8b7-4368-40e5-99e7-478534b5a036\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") "
Jul 07 15:00:07.387757 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387664    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-trusted-ca-bundle\") pod \"5a63e8b7-4368-40e5-99e7-478534b5a036\" (UID: \"5a63e8b7-4368-40e5-99e7-478534b5a036\") "
Jul 07 15:00:07.388149 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387882    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a63e8b7-4368-40e5-99e7-478534b5a036" (UID: "5a63e8b7-4368-40e5-99e7-478534b5a036"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:07.388149 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.387969    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-service-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:07.388149 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.388014    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-console-config" (OuterVolumeSpecName: "console-config") pod "5a63e8b7-4368-40e5-99e7-478534b5a036" (UID: "5a63e8b7-4368-40e5-99e7-478534b5a036"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:07.388356 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.388201    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a63e8b7-4368-40e5-99e7-478534b5a036" (UID: "5a63e8b7-4368-40e5-99e7-478534b5a036"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:07.388356 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.388227    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a63e8b7-4368-40e5-99e7-478534b5a036" (UID: "5a63e8b7-4368-40e5-99e7-478534b5a036"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:07.389540 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.389518    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a63e8b7-4368-40e5-99e7-478534b5a036-kube-api-access-lfk7z" (OuterVolumeSpecName: "kube-api-access-lfk7z") pod "5a63e8b7-4368-40e5-99e7-478534b5a036" (UID: "5a63e8b7-4368-40e5-99e7-478534b5a036"). InnerVolumeSpecName "kube-api-access-lfk7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:00:07.389916 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.389886    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a63e8b7-4368-40e5-99e7-478534b5a036" (UID: "5a63e8b7-4368-40e5-99e7-478534b5a036"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:00:07.390120 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.390096    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a63e8b7-4368-40e5-99e7-478534b5a036" (UID: "5a63e8b7-4368-40e5-99e7-478534b5a036"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:00:07.488858 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.488837    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-oauth-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:07.488967 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.488862    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-trusted-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:07.488967 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.488876    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfk7z\" (UniqueName: \"kubernetes.io/projected/5a63e8b7-4368-40e5-99e7-478534b5a036-kube-api-access-lfk7z\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:07.488967 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.488888    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-oauth-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:07.488967 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.488901    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a63e8b7-4368-40e5-99e7-478534b5a036-console-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:07.488967 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:07.488915    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a63e8b7-4368-40e5-99e7-478534b5a036-console-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:08.284322 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:08.284298    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d96ccc7cb-txh5r"
Jul 07 15:00:08.284720 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:08.284297    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d96ccc7cb-txh5r" event={"ID":"5a63e8b7-4368-40e5-99e7-478534b5a036","Type":"ContainerDied","Data":"a66310e58541e2a3dc21598075c1940f1e41c3cd34b328404ee3991f901b0fcb"}
Jul 07 15:00:08.284720 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:08.284430    2556 scope.go:117] "RemoveContainer" containerID="3486c1347dc2c9e331f36249aded63112c7ab9ed6ac290ce2533cc721b7ebfec"
Jul 07 15:00:08.307126 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:08.307100    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d96ccc7cb-txh5r"]
Jul 07 15:00:08.313382 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:08.313364    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d96ccc7cb-txh5r"]
Jul 07 15:00:09.269492 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:09.269458    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a63e8b7-4368-40e5-99e7-478534b5a036" path="/var/lib/kubelet/pods/5a63e8b7-4368-40e5-99e7-478534b5a036/volumes"
Jul 07 15:00:18.304148 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.304098    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5cf55649d4-92xck" podUID="37fbb372-18e2-40ce-a87d-af0da19a3530" containerName="console" containerID="cri-o://572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d" gracePeriod=15
Jul 07 15:00:18.555013 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.554952    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 15:00:18.666781 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.666754    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-oauth-config\") pod \"37fbb372-18e2-40ce-a87d-af0da19a3530\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") "
Jul 07 15:00:18.666912 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.666811    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-service-ca\") pod \"37fbb372-18e2-40ce-a87d-af0da19a3530\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") "
Jul 07 15:00:18.666912 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.666868    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-serving-cert\") pod \"37fbb372-18e2-40ce-a87d-af0da19a3530\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") "
Jul 07 15:00:18.667014 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.666913    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-console-config\") pod \"37fbb372-18e2-40ce-a87d-af0da19a3530\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") "
Jul 07 15:00:18.667014 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.666938    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qhl7\" (UniqueName: \"kubernetes.io/projected/37fbb372-18e2-40ce-a87d-af0da19a3530-kube-api-access-9qhl7\") pod \"37fbb372-18e2-40ce-a87d-af0da19a3530\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") "
Jul 07 15:00:18.667014 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.666981    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-trusted-ca-bundle\") pod \"37fbb372-18e2-40ce-a87d-af0da19a3530\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") "
Jul 07 15:00:18.667159 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.667025    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-oauth-serving-cert\") pod \"37fbb372-18e2-40ce-a87d-af0da19a3530\" (UID: \"37fbb372-18e2-40ce-a87d-af0da19a3530\") "
Jul 07 15:00:18.667308 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.667275    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-service-ca" (OuterVolumeSpecName: "service-ca") pod "37fbb372-18e2-40ce-a87d-af0da19a3530" (UID: "37fbb372-18e2-40ce-a87d-af0da19a3530"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:18.667388 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.667338    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-console-config" (OuterVolumeSpecName: "console-config") pod "37fbb372-18e2-40ce-a87d-af0da19a3530" (UID: "37fbb372-18e2-40ce-a87d-af0da19a3530"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:18.667617 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.667591    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "37fbb372-18e2-40ce-a87d-af0da19a3530" (UID: "37fbb372-18e2-40ce-a87d-af0da19a3530"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:18.667696 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.667597    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "37fbb372-18e2-40ce-a87d-af0da19a3530" (UID: "37fbb372-18e2-40ce-a87d-af0da19a3530"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:00:18.668985 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.668965    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "37fbb372-18e2-40ce-a87d-af0da19a3530" (UID: "37fbb372-18e2-40ce-a87d-af0da19a3530"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:00:18.669337 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.669311    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37fbb372-18e2-40ce-a87d-af0da19a3530-kube-api-access-9qhl7" (OuterVolumeSpecName: "kube-api-access-9qhl7") pod "37fbb372-18e2-40ce-a87d-af0da19a3530" (UID: "37fbb372-18e2-40ce-a87d-af0da19a3530"). InnerVolumeSpecName "kube-api-access-9qhl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:00:18.669337 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.669328    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "37fbb372-18e2-40ce-a87d-af0da19a3530" (UID: "37fbb372-18e2-40ce-a87d-af0da19a3530"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:00:18.768249 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.768225    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-trusted-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:18.768249 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.768244    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-oauth-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:18.768249 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.768253    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-oauth-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:18.768407 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.768262    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-service-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:18.768407 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.768270    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37fbb372-18e2-40ce-a87d-af0da19a3530-console-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:18.768407 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.768279    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37fbb372-18e2-40ce-a87d-af0da19a3530-console-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:18.768407 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:18.768288    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qhl7\" (UniqueName: \"kubernetes.io/projected/37fbb372-18e2-40ce-a87d-af0da19a3530-kube-api-access-9qhl7\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:19.330438 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.330406    2556 generic.go:356] "Generic (PLEG): container finished" podID="37fbb372-18e2-40ce-a87d-af0da19a3530" containerID="572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d" exitCode=0
Jul 07 15:00:19.330873 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.330464    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf55649d4-92xck"
Jul 07 15:00:19.330873 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.330471    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf55649d4-92xck" event={"ID":"37fbb372-18e2-40ce-a87d-af0da19a3530","Type":"ContainerDied","Data":"572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d"}
Jul 07 15:00:19.330873 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.330575    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf55649d4-92xck" event={"ID":"37fbb372-18e2-40ce-a87d-af0da19a3530","Type":"ContainerDied","Data":"3b7df46c932904a1e1e66a76bb502f0d65e265a98a664bd648fa666517082f47"}
Jul 07 15:00:19.330873 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.330598    2556 scope.go:117] "RemoveContainer" containerID="572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d"
Jul 07 15:00:19.338743 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.338727    2556 scope.go:117] "RemoveContainer" containerID="572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d"
Jul 07 15:00:19.339023 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:00:19.339003    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d\": container with ID starting with 572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d not found: ID does not exist" containerID="572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d"
Jul 07 15:00:19.339117 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.339029    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d"} err="failed to get container status \"572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d\": rpc error: code = NotFound desc = could not find container \"572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d\": container with ID starting with 572999f76d5b84f878a822483096af127d7746182b2bed8d154b4f0f6dcdf42d not found: ID does not exist"
Jul 07 15:00:19.348294 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.348271    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf55649d4-92xck"]
Jul 07 15:00:19.354173 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:19.354155    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cf55649d4-92xck"]
Jul 07 15:00:21.269388 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:21.269350    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37fbb372-18e2-40ce-a87d-af0da19a3530" path="/var/lib/kubelet/pods/37fbb372-18e2-40ce-a87d-af0da19a3530/volumes"
Jul 07 15:00:31.420843 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.420814    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"]
Jul 07 15:00:31.421203 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.421133    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37fbb372-18e2-40ce-a87d-af0da19a3530" containerName="console"
Jul 07 15:00:31.421203 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.421148    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="37fbb372-18e2-40ce-a87d-af0da19a3530" containerName="console"
Jul 07 15:00:31.421203 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.421162    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a63e8b7-4368-40e5-99e7-478534b5a036" containerName="console"
Jul 07 15:00:31.421203 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.421168    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63e8b7-4368-40e5-99e7-478534b5a036" containerName="console"
Jul 07 15:00:31.421321 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.421225    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a63e8b7-4368-40e5-99e7-478534b5a036" containerName="console"
Jul 07 15:00:31.421321 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.421238    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="37fbb372-18e2-40ce-a87d-af0da19a3530" containerName="console"
Jul 07 15:00:31.426065 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.426048    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.428519 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.428498    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 15:00:31.428691 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.428669    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 15:00:31.428749 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.428712    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-twnh6\""
Jul 07 15:00:31.433480 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.433457    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"]
Jul 07 15:00:31.566456 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.566425    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.566587 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.566472    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.566587 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.566556    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf7qd\" (UniqueName: \"kubernetes.io/projected/b5f201ec-361f-459a-ba05-b96ee2be1d9a-kube-api-access-wf7qd\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.667527 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.667496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf7qd\" (UniqueName: \"kubernetes.io/projected/b5f201ec-361f-459a-ba05-b96ee2be1d9a-kube-api-access-wf7qd\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.667655 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.667530    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.667655 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.667580    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.667878 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.667861    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.667961 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.667941    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.676511 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.676458    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf7qd\" (UniqueName: \"kubernetes.io/projected/b5f201ec-361f-459a-ba05-b96ee2be1d9a-kube-api-access-wf7qd\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.735404 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.735382    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:31.864471 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:31.864382    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"]
Jul 07 15:00:31.867124 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:00:31.867094    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f201ec_361f_459a_ba05_b96ee2be1d9a.slice/crio-767c1d05bff6093ef07deb17bb7de6e4a266d70bc1651339ad01d211aaf4651d WatchSource:0}: Error finding container 767c1d05bff6093ef07deb17bb7de6e4a266d70bc1651339ad01d211aaf4651d: Status 404 returned error can't find the container with id 767c1d05bff6093ef07deb17bb7de6e4a266d70bc1651339ad01d211aaf4651d
Jul 07 15:00:32.383058 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:32.383019    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2" event={"ID":"b5f201ec-361f-459a-ba05-b96ee2be1d9a","Type":"ContainerStarted","Data":"767c1d05bff6093ef07deb17bb7de6e4a266d70bc1651339ad01d211aaf4651d"}
Jul 07 15:00:41.426914 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:41.426877    2556 generic.go:356] "Generic (PLEG): container finished" podID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerID="3f2f090cb1f9693587191412a0cb3a4f5e8a9ee89613602c00a97486988bfe51" exitCode=0
Jul 07 15:00:41.427299 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:41.426960    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2" event={"ID":"b5f201ec-361f-459a-ba05-b96ee2be1d9a","Type":"ContainerDied","Data":"3f2f090cb1f9693587191412a0cb3a4f5e8a9ee89613602c00a97486988bfe51"}
Jul 07 15:00:45.447217 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:45.447176    2556 generic.go:356] "Generic (PLEG): container finished" podID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerID="2cf44382e010cc5b66596fead1c132feb6d3d70ded3267e283a4d4fda0c0ddc3" exitCode=0
Jul 07 15:00:45.447596 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:45.447264    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2" event={"ID":"b5f201ec-361f-459a-ba05-b96ee2be1d9a","Type":"ContainerDied","Data":"2cf44382e010cc5b66596fead1c132feb6d3d70ded3267e283a4d4fda0c0ddc3"}
Jul 07 15:00:53.484194 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:53.484163    2556 generic.go:356] "Generic (PLEG): container finished" podID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerID="27874747d2b8028a2f5dfcb5591afc3f3ad3e9e2b42466f9e42a3eb095da7d7a" exitCode=0
Jul 07 15:00:53.484528 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:53.484219    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2" event={"ID":"b5f201ec-361f-459a-ba05-b96ee2be1d9a","Type":"ContainerDied","Data":"27874747d2b8028a2f5dfcb5591afc3f3ad3e9e2b42466f9e42a3eb095da7d7a"}
Jul 07 15:00:54.610731 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.610709    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:54.751490 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.751431    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-util\") pod \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") "
Jul 07 15:00:54.751490 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.751473    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-bundle\") pod \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") "
Jul 07 15:00:54.751660 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.751494    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf7qd\" (UniqueName: \"kubernetes.io/projected/b5f201ec-361f-459a-ba05-b96ee2be1d9a-kube-api-access-wf7qd\") pod \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\" (UID: \"b5f201ec-361f-459a-ba05-b96ee2be1d9a\") "
Jul 07 15:00:54.752067 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.752043    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-bundle" (OuterVolumeSpecName: "bundle") pod "b5f201ec-361f-459a-ba05-b96ee2be1d9a" (UID: "b5f201ec-361f-459a-ba05-b96ee2be1d9a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:54.753649 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.753625    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f201ec-361f-459a-ba05-b96ee2be1d9a-kube-api-access-wf7qd" (OuterVolumeSpecName: "kube-api-access-wf7qd") pod "b5f201ec-361f-459a-ba05-b96ee2be1d9a" (UID: "b5f201ec-361f-459a-ba05-b96ee2be1d9a"). InnerVolumeSpecName "kube-api-access-wf7qd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:00:54.755672 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.755646    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-util" (OuterVolumeSpecName: "util") pod "b5f201ec-361f-459a-ba05-b96ee2be1d9a" (UID: "b5f201ec-361f-459a-ba05-b96ee2be1d9a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:54.852845 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.852822    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-util\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:54.852845 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.852843    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f201ec-361f-459a-ba05-b96ee2be1d9a-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:54.852977 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:54.852852    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf7qd\" (UniqueName: \"kubernetes.io/projected/b5f201ec-361f-459a-ba05-b96ee2be1d9a-kube-api-access-wf7qd\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:00:55.494161 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:55.494132    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2"
Jul 07 15:00:55.494161 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:55.494138    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589xs7f2" event={"ID":"b5f201ec-361f-459a-ba05-b96ee2be1d9a","Type":"ContainerDied","Data":"767c1d05bff6093ef07deb17bb7de6e4a266d70bc1651339ad01d211aaf4651d"}
Jul 07 15:00:55.494332 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:00:55.494186    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="767c1d05bff6093ef07deb17bb7de6e4a266d70bc1651339ad01d211aaf4651d"
Jul 07 15:01:00.148568 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148532    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"]
Jul 07 15:01:00.149069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148898    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerName="extract"
Jul 07 15:01:00.149069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148910    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerName="extract"
Jul 07 15:01:00.149069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148921    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerName="util"
Jul 07 15:01:00.149069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148928    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerName="util"
Jul 07 15:01:00.149069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148942    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerName="pull"
Jul 07 15:01:00.149069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148949    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerName="pull"
Jul 07 15:01:00.149069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.148998    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f201ec-361f-459a-ba05-b96ee2be1d9a" containerName="extract"
Jul 07 15:01:00.153736 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.153720    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.156470 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.156445    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 07 15:01:00.156470 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.156461    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 07 15:01:00.156749 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.156731    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-tjwfn\""
Jul 07 15:01:00.156969 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.156954    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 07 15:01:00.164316 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.164293    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"]
Jul 07 15:01:00.294929 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.294903    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/483a5a13-efe2-459a-a3ba-42daa7a8624e-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv\" (UID: \"483a5a13-efe2-459a-a3ba-42daa7a8624e\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.295033 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.294938    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vngqj\" (UniqueName: \"kubernetes.io/projected/483a5a13-efe2-459a-a3ba-42daa7a8624e-kube-api-access-vngqj\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv\" (UID: \"483a5a13-efe2-459a-a3ba-42daa7a8624e\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.395692 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.395669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/483a5a13-efe2-459a-a3ba-42daa7a8624e-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv\" (UID: \"483a5a13-efe2-459a-a3ba-42daa7a8624e\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.395823 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.395705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vngqj\" (UniqueName: \"kubernetes.io/projected/483a5a13-efe2-459a-a3ba-42daa7a8624e-kube-api-access-vngqj\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv\" (UID: \"483a5a13-efe2-459a-a3ba-42daa7a8624e\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.398045 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.398021    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/483a5a13-efe2-459a-a3ba-42daa7a8624e-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv\" (UID: \"483a5a13-efe2-459a-a3ba-42daa7a8624e\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.407672 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.407616    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vngqj\" (UniqueName: \"kubernetes.io/projected/483a5a13-efe2-459a-a3ba-42daa7a8624e-kube-api-access-vngqj\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv\" (UID: \"483a5a13-efe2-459a-a3ba-42daa7a8624e\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.464453 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.464426    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:00.596652 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:00.596625    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"]
Jul 07 15:01:00.598771 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:01:00.598739    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483a5a13_efe2_459a_a3ba_42daa7a8624e.slice/crio-4796d123fc326cb5c20d5a7487059a353e2f6070c9ba495b8026c5c1c236c355 WatchSource:0}: Error finding container 4796d123fc326cb5c20d5a7487059a353e2f6070c9ba495b8026c5c1c236c355: Status 404 returned error can't find the container with id 4796d123fc326cb5c20d5a7487059a353e2f6070c9ba495b8026c5c1c236c355
Jul 07 15:01:01.522072 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:01.522039    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv" event={"ID":"483a5a13-efe2-459a-a3ba-42daa7a8624e","Type":"ContainerStarted","Data":"4796d123fc326cb5c20d5a7487059a353e2f6070c9ba495b8026c5c1c236c355"}
Jul 07 15:01:04.433666 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.433636    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-6mns8"]
Jul 07 15:01:04.436098 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.436081    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.438944 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.438920    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-f8s48\""
Jul 07 15:01:04.439123 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.439105    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 07 15:01:04.439213 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.439135    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 07 15:01:04.458053 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.458030    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-6mns8"]
Jul 07 15:01:04.527380 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.527350    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.527486 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.527395    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snhns\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-kube-api-access-snhns\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.527486 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.527444    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/2730445c-0125-4c20-b60f-c65c59a21f19-cabundle0\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.538708 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.538675    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv" event={"ID":"483a5a13-efe2-459a-a3ba-42daa7a8624e","Type":"ContainerStarted","Data":"c1d2162baacf3ae946500476059a127088f3b6bd78e4348931a1a82cddda480d"}
Jul 07 15:01:04.538846 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.538816    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:04.570556 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.570518    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv" podStartSLOduration=1.591027837 podStartE2EDuration="4.570504786s" totalImagesPullingTime="2.979476949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:04.569200966 +0000 UTC m=+281.917683665" watchObservedRunningTime="2026-07-07 15:01:04.570504786 +0000 UTC m=+281.918987416"
Jul 07 15:01:04.628415 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.628394    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.628510 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.628425    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snhns\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-kube-api-access-snhns\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.628510 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.628475    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/2730445c-0125-4c20-b60f-c65c59a21f19-cabundle0\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.628595 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:04.628537    2556 secret.go:281] references non-existent secret key: ca.crt
Jul 07 15:01:04.628595 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:04.628553    2556 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 07 15:01:04.628595 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:04.628563    2556 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-6mns8: references non-existent secret key: ca.crt
Jul 07 15:01:04.628722 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:04.628599    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates podName:2730445c-0125-4c20-b60f-c65c59a21f19 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:05.128587743 +0000 UTC m=+282.477070352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates") pod "keda-operator-68bff59c-6mns8" (UID: "2730445c-0125-4c20-b60f-c65c59a21f19") : references non-existent secret key: ca.crt
Jul 07 15:01:04.629130 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.629111    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/2730445c-0125-4c20-b60f-c65c59a21f19-cabundle0\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.643497 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.643468    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snhns\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-kube-api-access-snhns\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:04.953442 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.953414    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"]
Jul 07 15:01:04.955712 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.955698    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:04.960211 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.960191    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 07 15:01:04.978680 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:04.978657    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"]
Jul 07 15:01:05.030966 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.030945    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xxjj\" (UniqueName: \"kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-kube-api-access-2xxjj\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.031041 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.030986    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-certificates\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.031041 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.031008    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/09819d22-a8a2-4dcf-8e59-acc92cee406d-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.131350 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.131330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:05.131436 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.131369    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xxjj\" (UniqueName: \"kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-kube-api-access-2xxjj\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.131436 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.131407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-certificates\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.131436 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.131427    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/09819d22-a8a2-4dcf-8e59-acc92cee406d-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.131532 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131449    2556 secret.go:281] references non-existent secret key: ca.crt
Jul 07 15:01:05.131532 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131464    2556 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 07 15:01:05.131532 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131472    2556 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-6mns8: references non-existent secret key: ca.crt
Jul 07 15:01:05.131532 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131525    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates podName:2730445c-0125-4c20-b60f-c65c59a21f19 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:06.131509654 +0000 UTC m=+283.479992282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates") pod "keda-operator-68bff59c-6mns8" (UID: "2730445c-0125-4c20-b60f-c65c59a21f19") : references non-existent secret key: ca.crt
Jul 07 15:01:05.131532 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131528    2556 secret.go:281] references non-existent secret key: tls.crt
Jul 07 15:01:05.131724 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131543    2556 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 07 15:01:05.131724 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131562    2556 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-v8c54: references non-existent secret key: tls.crt
Jul 07 15:01:05.131724 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:01:05.131605    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-certificates podName:09819d22-a8a2-4dcf-8e59-acc92cee406d nodeName:}" failed. No retries permitted until 2026-07-07 15:01:05.631593327 +0000 UTC m=+282.980075935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-certificates") pod "keda-metrics-apiserver-559f75f947-v8c54" (UID: "09819d22-a8a2-4dcf-8e59-acc92cee406d") : references non-existent secret key: tls.crt
Jul 07 15:01:05.131844 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.131755    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/09819d22-a8a2-4dcf-8e59-acc92cee406d-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.141334 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.141304    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xxjj\" (UniqueName: \"kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-kube-api-access-2xxjj\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.634744 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.634713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-certificates\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.637422 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.637398    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/09819d22-a8a2-4dcf-8e59-acc92cee406d-certificates\") pod \"keda-metrics-apiserver-559f75f947-v8c54\" (UID: \"09819d22-a8a2-4dcf-8e59-acc92cee406d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.866232 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.866209    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:05.994884 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:05.994860    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"]
Jul 07 15:01:05.997500 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:01:05.997472    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09819d22_a8a2_4dcf_8e59_acc92cee406d.slice/crio-1e6bf172635423f7a0090a3687a95b89e852c5079bd19db90ed9abf60177f2af WatchSource:0}: Error finding container 1e6bf172635423f7a0090a3687a95b89e852c5079bd19db90ed9abf60177f2af: Status 404 returned error can't find the container with id 1e6bf172635423f7a0090a3687a95b89e852c5079bd19db90ed9abf60177f2af
Jul 07 15:01:06.138704 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:06.138681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:06.140986 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:06.140969    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2730445c-0125-4c20-b60f-c65c59a21f19-certificates\") pod \"keda-operator-68bff59c-6mns8\" (UID: \"2730445c-0125-4c20-b60f-c65c59a21f19\") " pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:06.246066 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:06.246000    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:06.373239 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:06.373218    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-6mns8"]
Jul 07 15:01:06.375263 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:01:06.375232    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2730445c_0125_4c20_b60f_c65c59a21f19.slice/crio-e28cfdecc2effbabb6709f140ccc91793316582d332d38b826ad1b268af42bce WatchSource:0}: Error finding container e28cfdecc2effbabb6709f140ccc91793316582d332d38b826ad1b268af42bce: Status 404 returned error can't find the container with id e28cfdecc2effbabb6709f140ccc91793316582d332d38b826ad1b268af42bce
Jul 07 15:01:06.547493 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:06.547436    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54" event={"ID":"09819d22-a8a2-4dcf-8e59-acc92cee406d","Type":"ContainerStarted","Data":"1e6bf172635423f7a0090a3687a95b89e852c5079bd19db90ed9abf60177f2af"}
Jul 07 15:01:06.548488 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:06.548468    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-6mns8" event={"ID":"2730445c-0125-4c20-b60f-c65c59a21f19","Type":"ContainerStarted","Data":"e28cfdecc2effbabb6709f140ccc91793316582d332d38b826ad1b268af42bce"}
Jul 07 15:01:10.573988 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:10.573939    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54" event={"ID":"09819d22-a8a2-4dcf-8e59-acc92cee406d","Type":"ContainerStarted","Data":"35b3b6b04d240003fa9a747cb31ab9738158c6fdbdbb7aa97383a1c093d517ed"}
Jul 07 15:01:10.574428 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:10.574120    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:10.575606 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:10.575579    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-6mns8" event={"ID":"2730445c-0125-4c20-b60f-c65c59a21f19","Type":"ContainerStarted","Data":"c3dda8c81138604823cd908ffb0159d98905f2f5527a1af5cc598e312a962e92"}
Jul 07 15:01:10.575718 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:10.575640    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:01:10.592403 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:10.592359    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54" podStartSLOduration=2.75117023 podStartE2EDuration="6.592345443s" totalImagesPullingTime="3.841175213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:10.590833869 +0000 UTC m=+287.939316500" watchObservedRunningTime="2026-07-07 15:01:10.592345443 +0000 UTC m=+287.940828073"
Jul 07 15:01:10.610283 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:10.610240    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-6mns8" podStartSLOduration=3.141300651 podStartE2EDuration="6.61022766s" totalImagesPullingTime="3.468927009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:10.609108069 +0000 UTC m=+287.957590718" watchObservedRunningTime="2026-07-07 15:01:10.61022766 +0000 UTC m=+287.958710290"
Jul 07 15:01:21.585243 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:21.585208    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-v8c54"
Jul 07 15:01:23.091724 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:23.091691    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:01:23.097941 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:23.097912    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:01:23.103969 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:23.103948    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 15:01:23.106921 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:23.106900    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:01:23.113315 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:23.113279    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:01:25.544687 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:25.544657    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-ghjlv"
Jul 07 15:01:31.582433 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:01:31.582407    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-6mns8"
Jul 07 15:02:14.694664 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.694601    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-hbpmh"]
Jul 07 15:02:14.697026 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.697009    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:14.700914 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.700886    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 07 15:02:14.701031 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.700916    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 07 15:02:14.701031 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.700898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-knj7r\""
Jul 07 15:02:14.701031 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.700952    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 07 15:02:14.724723 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.724698    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-hbpmh"]
Jul 07 15:02:14.760788 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.760759    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-6vmr2"]
Jul 07 15:02:14.763087 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.763071    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:14.765775 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.765755    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 07 15:02:14.765921 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.765777    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-lsd7p\""
Jul 07 15:02:14.777689 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.777668    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-6vmr2"]
Jul 07 15:02:14.870675 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.870646    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrmlf\" (UniqueName: \"kubernetes.io/projected/924ebc4e-458f-47d3-8d66-4dd15d9e3103-kube-api-access-jrmlf\") pod \"seaweedfs-64568bcd49-6vmr2\" (UID: \"924ebc4e-458f-47d3-8d66-4dd15d9e3103\") " pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:14.870858 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.870679    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad16c964-8f85-4043-908b-4a6db81005dc-cert\") pod \"kserve-controller-manager-666b75c8f7-hbpmh\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") " pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:14.870858 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.870811    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97klc\" (UniqueName: \"kubernetes.io/projected/ad16c964-8f85-4043-908b-4a6db81005dc-kube-api-access-97klc\") pod \"kserve-controller-manager-666b75c8f7-hbpmh\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") " pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:14.870980 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.870877    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/924ebc4e-458f-47d3-8d66-4dd15d9e3103-data\") pod \"seaweedfs-64568bcd49-6vmr2\" (UID: \"924ebc4e-458f-47d3-8d66-4dd15d9e3103\") " pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:14.971384 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.971271    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97klc\" (UniqueName: \"kubernetes.io/projected/ad16c964-8f85-4043-908b-4a6db81005dc-kube-api-access-97klc\") pod \"kserve-controller-manager-666b75c8f7-hbpmh\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") " pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:14.971384 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.971312    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/924ebc4e-458f-47d3-8d66-4dd15d9e3103-data\") pod \"seaweedfs-64568bcd49-6vmr2\" (UID: \"924ebc4e-458f-47d3-8d66-4dd15d9e3103\") " pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:14.971593 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.971436    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrmlf\" (UniqueName: \"kubernetes.io/projected/924ebc4e-458f-47d3-8d66-4dd15d9e3103-kube-api-access-jrmlf\") pod \"seaweedfs-64568bcd49-6vmr2\" (UID: \"924ebc4e-458f-47d3-8d66-4dd15d9e3103\") " pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:14.971593 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.971473    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad16c964-8f85-4043-908b-4a6db81005dc-cert\") pod \"kserve-controller-manager-666b75c8f7-hbpmh\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") " pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:14.971719 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.971639    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/924ebc4e-458f-47d3-8d66-4dd15d9e3103-data\") pod \"seaweedfs-64568bcd49-6vmr2\" (UID: \"924ebc4e-458f-47d3-8d66-4dd15d9e3103\") " pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:14.973942 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.973925    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad16c964-8f85-4043-908b-4a6db81005dc-cert\") pod \"kserve-controller-manager-666b75c8f7-hbpmh\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") " pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:14.981267 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.981241    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97klc\" (UniqueName: \"kubernetes.io/projected/ad16c964-8f85-4043-908b-4a6db81005dc-kube-api-access-97klc\") pod \"kserve-controller-manager-666b75c8f7-hbpmh\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") " pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:14.981372 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:14.981271    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrmlf\" (UniqueName: \"kubernetes.io/projected/924ebc4e-458f-47d3-8d66-4dd15d9e3103-kube-api-access-jrmlf\") pod \"seaweedfs-64568bcd49-6vmr2\" (UID: \"924ebc4e-458f-47d3-8d66-4dd15d9e3103\") " pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:15.007122 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:15.007096    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:15.072668 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:15.072638    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:15.157949 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:15.157513    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-hbpmh"]
Jul 07 15:02:15.161104 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:02:15.161066    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad16c964_8f85_4043_908b_4a6db81005dc.slice/crio-2ada5e461780926cf8d4b6deae8f0fe0e64422f224ca78a6c799d1d429a3e73e WatchSource:0}: Error finding container 2ada5e461780926cf8d4b6deae8f0fe0e64422f224ca78a6c799d1d429a3e73e: Status 404 returned error can't find the container with id 2ada5e461780926cf8d4b6deae8f0fe0e64422f224ca78a6c799d1d429a3e73e
Jul 07 15:02:15.162420 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:15.162395    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:02:15.211892 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:15.211701    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-6vmr2"]
Jul 07 15:02:15.214385 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:02:15.214355    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924ebc4e_458f_47d3_8d66_4dd15d9e3103.slice/crio-e805eb44d58ee92ee0b5854438371413717e506d8c73659c9a713ec76e325762 WatchSource:0}: Error finding container e805eb44d58ee92ee0b5854438371413717e506d8c73659c9a713ec76e325762: Status 404 returned error can't find the container with id e805eb44d58ee92ee0b5854438371413717e506d8c73659c9a713ec76e325762
Jul 07 15:02:15.889176 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:15.889141    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-6vmr2" event={"ID":"924ebc4e-458f-47d3-8d66-4dd15d9e3103","Type":"ContainerStarted","Data":"e805eb44d58ee92ee0b5854438371413717e506d8c73659c9a713ec76e325762"}
Jul 07 15:02:15.890435 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:15.890407    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh" event={"ID":"ad16c964-8f85-4043-908b-4a6db81005dc","Type":"ContainerStarted","Data":"2ada5e461780926cf8d4b6deae8f0fe0e64422f224ca78a6c799d1d429a3e73e"}
Jul 07 15:02:19.911880 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:19.911847    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-6vmr2" event={"ID":"924ebc4e-458f-47d3-8d66-4dd15d9e3103","Type":"ContainerStarted","Data":"cc7c8d1754e9976dbd2bca15907ae0638140414e520dc61635ecd25361c55277"}
Jul 07 15:02:19.912307 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:19.911932    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:19.913426 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:19.913396    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh" event={"ID":"ad16c964-8f85-4043-908b-4a6db81005dc","Type":"ContainerStarted","Data":"4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09"}
Jul 07 15:02:19.913555 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:19.913445    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:19.932178 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:19.932133    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-6vmr2" podStartSLOduration=1.526702474 podStartE2EDuration="5.93211795s" totalImagesPullingTime="4.405415476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:19.930838193 +0000 UTC m=+357.279320842" watchObservedRunningTime="2026-07-07 15:02:19.93211795 +0000 UTC m=+357.280600580"
Jul 07 15:02:19.951617 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:19.951567    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh" podStartSLOduration=1.580752038 podStartE2EDuration="5.951550875s" totalImagesPullingTime="4.370798837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:19.949952096 +0000 UTC m=+357.298434744" watchObservedRunningTime="2026-07-07 15:02:19.951550875 +0000 UTC m=+357.300033559"
Jul 07 15:02:25.921111 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:25.921075    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-6vmr2"
Jul 07 15:02:50.923780 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:50.923747    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:55.994332 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:55.994298    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-hbpmh"]
Jul 07 15:02:55.994815 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:55.994483    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh" podUID="ad16c964-8f85-4043-908b-4a6db81005dc" containerName="manager" containerID="cri-o://4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09" gracePeriod=10
Jul 07 15:02:56.028976 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.028948    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-9q6c4"]
Jul 07 15:02:56.031131 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.031115    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.045283 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.045258    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-9q6c4"]
Jul 07 15:02:56.125596 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.125573    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b17b94b1-9a20-46ea-a1dc-75d2d01b4abb-cert\") pod \"kserve-controller-manager-666b75c8f7-9q6c4\" (UID: \"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb\") " pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.125713 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.125611    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24hfq\" (UniqueName: \"kubernetes.io/projected/b17b94b1-9a20-46ea-a1dc-75d2d01b4abb-kube-api-access-24hfq\") pod \"kserve-controller-manager-666b75c8f7-9q6c4\" (UID: \"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb\") " pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.227240 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.227213    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b17b94b1-9a20-46ea-a1dc-75d2d01b4abb-cert\") pod \"kserve-controller-manager-666b75c8f7-9q6c4\" (UID: \"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb\") " pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.227381 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.227256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24hfq\" (UniqueName: \"kubernetes.io/projected/b17b94b1-9a20-46ea-a1dc-75d2d01b4abb-kube-api-access-24hfq\") pod \"kserve-controller-manager-666b75c8f7-9q6c4\" (UID: \"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb\") " pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.229543 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.229523    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b17b94b1-9a20-46ea-a1dc-75d2d01b4abb-cert\") pod \"kserve-controller-manager-666b75c8f7-9q6c4\" (UID: \"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb\") " pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.235459 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.235434    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24hfq\" (UniqueName: \"kubernetes.io/projected/b17b94b1-9a20-46ea-a1dc-75d2d01b4abb-kube-api-access-24hfq\") pod \"kserve-controller-manager-666b75c8f7-9q6c4\" (UID: \"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb\") " pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.253285 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.253234    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:56.379831 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.379776    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:56.429038 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.429012    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad16c964-8f85-4043-908b-4a6db81005dc-cert\") pod \"ad16c964-8f85-4043-908b-4a6db81005dc\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") "
Jul 07 15:02:56.429293 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.429269    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97klc\" (UniqueName: \"kubernetes.io/projected/ad16c964-8f85-4043-908b-4a6db81005dc-kube-api-access-97klc\") pod \"ad16c964-8f85-4043-908b-4a6db81005dc\" (UID: \"ad16c964-8f85-4043-908b-4a6db81005dc\") "
Jul 07 15:02:56.433108 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.433072    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad16c964-8f85-4043-908b-4a6db81005dc-cert" (OuterVolumeSpecName: "cert") pod "ad16c964-8f85-4043-908b-4a6db81005dc" (UID: "ad16c964-8f85-4043-908b-4a6db81005dc"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:02:56.433211 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.433103    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad16c964-8f85-4043-908b-4a6db81005dc-kube-api-access-97klc" (OuterVolumeSpecName: "kube-api-access-97klc") pod "ad16c964-8f85-4043-908b-4a6db81005dc" (UID: "ad16c964-8f85-4043-908b-4a6db81005dc"). InnerVolumeSpecName "kube-api-access-97klc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:02:56.515119 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.515042    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-9q6c4"]
Jul 07 15:02:56.517925 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:02:56.517897    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb17b94b1_9a20_46ea_a1dc_75d2d01b4abb.slice/crio-3938ba7fc7a822ca5c2a4bfe86760e6277bbf018a5691d9caa7be30798b4d618 WatchSource:0}: Error finding container 3938ba7fc7a822ca5c2a4bfe86760e6277bbf018a5691d9caa7be30798b4d618: Status 404 returned error can't find the container with id 3938ba7fc7a822ca5c2a4bfe86760e6277bbf018a5691d9caa7be30798b4d618
Jul 07 15:02:56.530303 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.530278    2556 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad16c964-8f85-4043-908b-4a6db81005dc-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:02:56.530303 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:56.530301    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97klc\" (UniqueName: \"kubernetes.io/projected/ad16c964-8f85-4043-908b-4a6db81005dc-kube-api-access-97klc\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:02:57.081758 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.081725    2556 generic.go:356] "Generic (PLEG): container finished" podID="ad16c964-8f85-4043-908b-4a6db81005dc" containerID="4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09" exitCode=0
Jul 07 15:02:57.082160 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.081783    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh"
Jul 07 15:02:57.082160 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.081830    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh" event={"ID":"ad16c964-8f85-4043-908b-4a6db81005dc","Type":"ContainerDied","Data":"4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09"}
Jul 07 15:02:57.082160 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.081912    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-hbpmh" event={"ID":"ad16c964-8f85-4043-908b-4a6db81005dc","Type":"ContainerDied","Data":"2ada5e461780926cf8d4b6deae8f0fe0e64422f224ca78a6c799d1d429a3e73e"}
Jul 07 15:02:57.082160 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.081955    2556 scope.go:117] "RemoveContainer" containerID="4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09"
Jul 07 15:02:57.083313 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.083221    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4" event={"ID":"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb","Type":"ContainerStarted","Data":"3938ba7fc7a822ca5c2a4bfe86760e6277bbf018a5691d9caa7be30798b4d618"}
Jul 07 15:02:57.091213 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.091194    2556 scope.go:117] "RemoveContainer" containerID="4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09"
Jul 07 15:02:57.091495 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:02:57.091469    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09\": container with ID starting with 4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09 not found: ID does not exist" containerID="4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09"
Jul 07 15:02:57.091610 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.091503    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09"} err="failed to get container status \"4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09\": rpc error: code = NotFound desc = could not find container \"4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09\": container with ID starting with 4092db529ba37317a4e2d1b47ab0a1396bfdfa886e568112b14af3d3a9578a09 not found: ID does not exist"
Jul 07 15:02:57.104430 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.104409    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-hbpmh"]
Jul 07 15:02:57.112370 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.112339    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-hbpmh"]
Jul 07 15:02:57.270339 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:57.270282    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad16c964-8f85-4043-908b-4a6db81005dc" path="/var/lib/kubelet/pods/ad16c964-8f85-4043-908b-4a6db81005dc/volumes"
Jul 07 15:02:58.090415 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:58.090379    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4" event={"ID":"b17b94b1-9a20-46ea-a1dc-75d2d01b4abb","Type":"ContainerStarted","Data":"16bf78d86384ac52490d5a5d220e61da6e5f6418daa2005f649ddc48c1896001"}
Jul 07 15:02:58.090840 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:58.090463    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:02:58.109764 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:02:58.109715    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4" podStartSLOduration=1.29739285 podStartE2EDuration="2.109699103s" totalImagesPullingTime="812.306253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:58.108322829 +0000 UTC m=+395.456805473" watchObservedRunningTime="2026-07-07 15:02:58.109699103 +0000 UTC m=+395.458181733"
Jul 07 15:03:29.100702 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:29.100666    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-666b75c8f7-9q6c4"
Jul 07 15:03:30.224303 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.224272    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lc964"]
Jul 07 15:03:30.224665 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.224614    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad16c964-8f85-4043-908b-4a6db81005dc" containerName="manager"
Jul 07 15:03:30.224665 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.224625    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad16c964-8f85-4043-908b-4a6db81005dc" containerName="manager"
Jul 07 15:03:30.224734 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.224701    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad16c964-8f85-4043-908b-4a6db81005dc" containerName="manager"
Jul 07 15:03:30.227768 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.227752    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.230069 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.230047    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 07 15:03:30.230202 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.230081    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-6qhmj\""
Jul 07 15:03:30.237522 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.237498    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lc964"]
Jul 07 15:03:30.296344 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.296318    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70ce05a7-b0f4-47cd-b017-c872f8054b2b-cert\") pod \"odh-model-controller-84b54b7f97-lc964\" (UID: \"70ce05a7-b0f4-47cd-b017-c872f8054b2b\") " pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.296448 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.296353    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbgz\" (UniqueName: \"kubernetes.io/projected/70ce05a7-b0f4-47cd-b017-c872f8054b2b-kube-api-access-msbgz\") pod \"odh-model-controller-84b54b7f97-lc964\" (UID: \"70ce05a7-b0f4-47cd-b017-c872f8054b2b\") " pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.397321 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.397298    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70ce05a7-b0f4-47cd-b017-c872f8054b2b-cert\") pod \"odh-model-controller-84b54b7f97-lc964\" (UID: \"70ce05a7-b0f4-47cd-b017-c872f8054b2b\") " pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.397440 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.397338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbgz\" (UniqueName: \"kubernetes.io/projected/70ce05a7-b0f4-47cd-b017-c872f8054b2b-kube-api-access-msbgz\") pod \"odh-model-controller-84b54b7f97-lc964\" (UID: \"70ce05a7-b0f4-47cd-b017-c872f8054b2b\") " pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.399639 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.399614    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70ce05a7-b0f4-47cd-b017-c872f8054b2b-cert\") pod \"odh-model-controller-84b54b7f97-lc964\" (UID: \"70ce05a7-b0f4-47cd-b017-c872f8054b2b\") " pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.407322 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.407302    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbgz\" (UniqueName: \"kubernetes.io/projected/70ce05a7-b0f4-47cd-b017-c872f8054b2b-kube-api-access-msbgz\") pod \"odh-model-controller-84b54b7f97-lc964\" (UID: \"70ce05a7-b0f4-47cd-b017-c872f8054b2b\") " pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.539398 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.539339    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:30.668601 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:30.668578    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lc964"]
Jul 07 15:03:30.671428 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:03:30.671404    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ce05a7_b0f4_47cd_b017_c872f8054b2b.slice/crio-bf4c85d3404eb8b1361f9043d5d94c2cbe82837a851208ccf9b70f8d0c2c8081 WatchSource:0}: Error finding container bf4c85d3404eb8b1361f9043d5d94c2cbe82837a851208ccf9b70f8d0c2c8081: Status 404 returned error can't find the container with id bf4c85d3404eb8b1361f9043d5d94c2cbe82837a851208ccf9b70f8d0c2c8081
Jul 07 15:03:31.243052 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:31.243024    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-lc964" event={"ID":"70ce05a7-b0f4-47cd-b017-c872f8054b2b","Type":"ContainerStarted","Data":"bf4c85d3404eb8b1361f9043d5d94c2cbe82837a851208ccf9b70f8d0c2c8081"}
Jul 07 15:03:34.259339 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:34.259304    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-lc964" event={"ID":"70ce05a7-b0f4-47cd-b017-c872f8054b2b","Type":"ContainerStarted","Data":"f84e99500f64b504c418ac92ceda46ecb19dca0a6c251aa6e722aa4c70154b94"}
Jul 07 15:03:34.259728 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:34.259381    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:03:34.290894 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:34.284688    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-lc964" podStartSLOduration=1.108597276 podStartE2EDuration="4.284671315s" totalImagesPullingTime="3.176074039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:03:34.282411367 +0000 UTC m=+431.630894000" watchObservedRunningTime="2026-07-07 15:03:34.284671315 +0000 UTC m=+431.633153946"
Jul 07 15:03:45.270315 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:03:45.270239    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-lc964"
Jul 07 15:05:19.605943 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.605842    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"]
Jul 07 15:05:19.609679 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.609656    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.613645 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.613622    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f85ce-kube-rbac-proxy-sar-config\""
Jul 07 15:05:19.614805 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.614763    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 07 15:05:19.614908 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.614829    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gzwhn\""
Jul 07 15:05:19.614908 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.614846    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f85ce-predictor-serving-cert\""
Jul 07 15:05:19.615019 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.614845    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 15:05:19.623976 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.623955    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"]
Jul 07 15:05:19.724123 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.724089    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4qhs\" (UniqueName: \"kubernetes.io/projected/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-api-access-t4qhs\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.724292 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.724148    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.724292 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.724177    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2448e304-2d49-4f37-bdf8-f5598d6e4698-proxy-tls\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.824856 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.824786    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4qhs\" (UniqueName: \"kubernetes.io/projected/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-api-access-t4qhs\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.825047 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.824904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.825047 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.824945    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2448e304-2d49-4f37-bdf8-f5598d6e4698-proxy-tls\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.825735 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.825701    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.827623 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.827597    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2448e304-2d49-4f37-bdf8-f5598d6e4698-proxy-tls\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.838939 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.838907    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4qhs\" (UniqueName: \"kubernetes.io/projected/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-api-access-t4qhs\") pod \"success-200-isvc-f85ce-predictor-6578cf6d78-c57pb\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") " pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:19.919946 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:19.919912    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:20.069251 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:20.069212    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"]
Jul 07 15:05:20.071293 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:05:20.071252    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2448e304_2d49_4f37_bdf8_f5598d6e4698.slice/crio-b3b9e15790b4a094c2226314074a8b6e5d312bd4f364c1f0125ab37ac3e9965b WatchSource:0}: Error finding container b3b9e15790b4a094c2226314074a8b6e5d312bd4f364c1f0125ab37ac3e9965b: Status 404 returned error can't find the container with id b3b9e15790b4a094c2226314074a8b6e5d312bd4f364c1f0125ab37ac3e9965b
Jul 07 15:05:20.758227 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:20.758191    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" event={"ID":"2448e304-2d49-4f37-bdf8-f5598d6e4698","Type":"ContainerStarted","Data":"b3b9e15790b4a094c2226314074a8b6e5d312bd4f364c1f0125ab37ac3e9965b"}
Jul 07 15:05:33.834414 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:33.834328    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" event={"ID":"2448e304-2d49-4f37-bdf8-f5598d6e4698","Type":"ContainerStarted","Data":"689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a"}
Jul 07 15:05:36.852805 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:36.852764    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" event={"ID":"2448e304-2d49-4f37-bdf8-f5598d6e4698","Type":"ContainerStarted","Data":"617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1"}
Jul 07 15:05:36.853253 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:36.852843    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:36.853253 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:36.852890    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:36.854403 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:36.854376    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 15:05:36.873622 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:36.873568    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podStartSLOduration=2.100829717 podStartE2EDuration="17.873554853s" totalImagesPullingTime="15.772725136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:36.872058208 +0000 UTC m=+554.220540839" watchObservedRunningTime="2026-07-07 15:05:36.873554853 +0000 UTC m=+554.222037483"
Jul 07 15:05:37.857826 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:37.857767    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 15:05:42.862391 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:42.862360    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:05:42.862889 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:42.862862    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 15:05:52.863834 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:05:52.863757    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 15:06:02.863839 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:02.863775    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 15:06:12.863301 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:12.863260    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 15:06:22.864757 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:22.864728    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:06:23.126692 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:23.126615    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:06:23.132326 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:23.132304    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:06:23.145055 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:23.145028    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:06:23.150350 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:23.150332    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:06:39.577809 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.577750    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"]
Jul 07 15:06:39.580387 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.580371    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:39.583187 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.583166    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-f85ce-kube-rbac-proxy-sar-config\""
Jul 07 15:06:39.583187 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.583175    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-f85ce-serving-cert\""
Jul 07 15:06:39.592395 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.592371    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"]
Jul 07 15:06:39.638485 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.638458    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf7b141-0758-4ef9-85f0-a69102495a37-openshift-service-ca-bundle\") pod \"switch-graph-f85ce-7d456c44cd-fj847\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") " pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:39.638606 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.638514    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls\") pod \"switch-graph-f85ce-7d456c44cd-fj847\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") " pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:39.739501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.739463    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf7b141-0758-4ef9-85f0-a69102495a37-openshift-service-ca-bundle\") pod \"switch-graph-f85ce-7d456c44cd-fj847\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") " pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:39.739660 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.739525    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls\") pod \"switch-graph-f85ce-7d456c44cd-fj847\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") " pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:39.739660 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:06:39.739616    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-f85ce-serving-cert: secret "switch-graph-f85ce-serving-cert" not found
Jul 07 15:06:39.739731 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:06:39.739694    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls podName:9cf7b141-0758-4ef9-85f0-a69102495a37 nodeName:}" failed. No retries permitted until 2026-07-07 15:06:40.239677608 +0000 UTC m=+617.588160216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls") pod "switch-graph-f85ce-7d456c44cd-fj847" (UID: "9cf7b141-0758-4ef9-85f0-a69102495a37") : secret "switch-graph-f85ce-serving-cert" not found
Jul 07 15:06:39.740109 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:39.740090    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf7b141-0758-4ef9-85f0-a69102495a37-openshift-service-ca-bundle\") pod \"switch-graph-f85ce-7d456c44cd-fj847\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") " pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:40.243277 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:40.243242    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls\") pod \"switch-graph-f85ce-7d456c44cd-fj847\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") " pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:40.245735 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:40.245709    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls\") pod \"switch-graph-f85ce-7d456c44cd-fj847\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") " pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:40.490698 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:40.490666    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:40.625828 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:40.625779    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"]
Jul 07 15:06:40.628844 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:06:40.628780    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf7b141_0758_4ef9_85f0_a69102495a37.slice/crio-e3b206e3f78b1bba08924dea7a318779736716417abbe11d83087bfba4a6c3c4 WatchSource:0}: Error finding container e3b206e3f78b1bba08924dea7a318779736716417abbe11d83087bfba4a6c3c4: Status 404 returned error can't find the container with id e3b206e3f78b1bba08924dea7a318779736716417abbe11d83087bfba4a6c3c4
Jul 07 15:06:41.152182 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:41.152149    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" event={"ID":"9cf7b141-0758-4ef9-85f0-a69102495a37","Type":"ContainerStarted","Data":"e3b206e3f78b1bba08924dea7a318779736716417abbe11d83087bfba4a6c3c4"}
Jul 07 15:06:46.179114 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:46.179074    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" event={"ID":"9cf7b141-0758-4ef9-85f0-a69102495a37","Type":"ContainerStarted","Data":"463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9"}
Jul 07 15:06:46.179714 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:46.179193    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:46.207189 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:46.207139    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podStartSLOduration=2.296303788 podStartE2EDuration="7.207124007s" totalImagesPullingTime="4.910820219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:46.206152847 +0000 UTC m=+623.554635478" watchObservedRunningTime="2026-07-07 15:06:46.207124007 +0000 UTC m=+623.555606709"
Jul 07 15:06:52.189678 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:52.189647    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:06:53.843742 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:53.843709    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"]
Jul 07 15:06:53.844185 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:53.843896    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce" containerID="cri-o://463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9" gracePeriod=30
Jul 07 15:06:54.166716 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.166680    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"]
Jul 07 15:06:54.170621 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.170596    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.175033 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.175005    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3c9c9-kube-rbac-proxy-sar-config\""
Jul 07 15:06:54.175177 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.175009    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3c9c9-predictor-serving-cert\""
Jul 07 15:06:54.187595 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.187567    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"]
Jul 07 15:06:54.261245 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.261197    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvn6s\" (UniqueName: \"kubernetes.io/projected/df197fac-db99-4475-b945-3b9f0073656d-kube-api-access-tvn6s\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.261394 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.261331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df197fac-db99-4475-b945-3b9f0073656d-proxy-tls\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.261394 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.261356    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df197fac-db99-4475-b945-3b9f0073656d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.339622 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.339587    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"]
Jul 07 15:06:54.339836 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.339809    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container" containerID="cri-o://689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a" gracePeriod=30
Jul 07 15:06:54.339929 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.339843    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kube-rbac-proxy" containerID="cri-o://617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1" gracePeriod=30
Jul 07 15:06:54.362092 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.362069    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df197fac-db99-4475-b945-3b9f0073656d-proxy-tls\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.362202 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.362099    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df197fac-db99-4475-b945-3b9f0073656d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.362202 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.362160    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvn6s\" (UniqueName: \"kubernetes.io/projected/df197fac-db99-4475-b945-3b9f0073656d-kube-api-access-tvn6s\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.362993 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.362969    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df197fac-db99-4475-b945-3b9f0073656d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.364858 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.364834    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df197fac-db99-4475-b945-3b9f0073656d-proxy-tls\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.371943 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.371916    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvn6s\" (UniqueName: \"kubernetes.io/projected/df197fac-db99-4475-b945-3b9f0073656d-kube-api-access-tvn6s\") pod \"success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") " pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.484682 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.484654    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:54.625554 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:54.625530    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"]
Jul 07 15:06:54.627554 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:06:54.627522    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf197fac_db99_4475_b945_3b9f0073656d.slice/crio-2bd3510040be4b1685e6c02be7481293f0aa126ec00d9a1a952df5e459e3a87d WatchSource:0}: Error finding container 2bd3510040be4b1685e6c02be7481293f0aa126ec00d9a1a952df5e459e3a87d: Status 404 returned error can't find the container with id 2bd3510040be4b1685e6c02be7481293f0aa126ec00d9a1a952df5e459e3a87d
Jul 07 15:06:55.223136 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:55.223101    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" event={"ID":"df197fac-db99-4475-b945-3b9f0073656d","Type":"ContainerStarted","Data":"9ceae840324d4568616c83d7cde38aa601635bd492424d279dc1130d77cd689d"}
Jul 07 15:06:55.223581 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:55.223174    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:55.223581 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:55.223208    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" event={"ID":"df197fac-db99-4475-b945-3b9f0073656d","Type":"ContainerStarted","Data":"cc83f8c0f1a9869498c6e6c6bebdb36ceb86b9f50132ba150f52c5a8dab934ee"}
Jul 07 15:06:55.223581 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:55.223236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" event={"ID":"df197fac-db99-4475-b945-3b9f0073656d","Type":"ContainerStarted","Data":"2bd3510040be4b1685e6c02be7481293f0aa126ec00d9a1a952df5e459e3a87d"}
Jul 07 15:06:55.224973 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:55.224947    2556 generic.go:356] "Generic (PLEG): container finished" podID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerID="617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1" exitCode=2
Jul 07 15:06:55.225074 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:55.224993    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" event={"ID":"2448e304-2d49-4f37-bdf8-f5598d6e4698","Type":"ContainerDied","Data":"617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1"}
Jul 07 15:06:55.246696 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:55.246622    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podStartSLOduration=1.24660474 podStartE2EDuration="1.24660474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:55.245947021 +0000 UTC m=+632.594429671" watchObservedRunningTime="2026-07-07 15:06:55.24660474 +0000 UTC m=+632.595087371"
Jul 07 15:06:56.229853 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:56.229815    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:06:56.231413 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:56.231381    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 15:06:57.187210 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.187178    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:06:57.235364 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.235330    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 15:06:57.395856 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.395830    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:06:57.489988 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.489913    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2448e304-2d49-4f37-bdf8-f5598d6e4698-proxy-tls\") pod \"2448e304-2d49-4f37-bdf8-f5598d6e4698\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") "
Jul 07 15:06:57.489988 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.489980    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4qhs\" (UniqueName: \"kubernetes.io/projected/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-api-access-t4qhs\") pod \"2448e304-2d49-4f37-bdf8-f5598d6e4698\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") "
Jul 07 15:06:57.490185 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.490046    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-rbac-proxy-sar-config\") pod \"2448e304-2d49-4f37-bdf8-f5598d6e4698\" (UID: \"2448e304-2d49-4f37-bdf8-f5598d6e4698\") "
Jul 07 15:06:57.490421 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.490395    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2448e304-2d49-4f37-bdf8-f5598d6e4698" (UID: "2448e304-2d49-4f37-bdf8-f5598d6e4698"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:06:57.492142 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.492110    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2448e304-2d49-4f37-bdf8-f5598d6e4698-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2448e304-2d49-4f37-bdf8-f5598d6e4698" (UID: "2448e304-2d49-4f37-bdf8-f5598d6e4698"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:06:57.492142 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.492124    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-api-access-t4qhs" (OuterVolumeSpecName: "kube-api-access-t4qhs") pod "2448e304-2d49-4f37-bdf8-f5598d6e4698" (UID: "2448e304-2d49-4f37-bdf8-f5598d6e4698"). InnerVolumeSpecName "kube-api-access-t4qhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:06:57.590849 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.590815    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2448e304-2d49-4f37-bdf8-f5598d6e4698-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:06:57.590849 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.590846    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4qhs\" (UniqueName: \"kubernetes.io/projected/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-api-access-t4qhs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:06:57.591010 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:57.590860    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2448e304-2d49-4f37-bdf8-f5598d6e4698-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:06:58.241959 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.241926    2556 generic.go:356] "Generic (PLEG): container finished" podID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerID="689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a" exitCode=0
Jul 07 15:06:58.242396 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.241974    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"
Jul 07 15:06:58.242396 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.241977    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" event={"ID":"2448e304-2d49-4f37-bdf8-f5598d6e4698","Type":"ContainerDied","Data":"689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a"}
Jul 07 15:06:58.242396 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.242026    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb" event={"ID":"2448e304-2d49-4f37-bdf8-f5598d6e4698","Type":"ContainerDied","Data":"b3b9e15790b4a094c2226314074a8b6e5d312bd4f364c1f0125ab37ac3e9965b"}
Jul 07 15:06:58.242396 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.242052    2556 scope.go:117] "RemoveContainer" containerID="617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1"
Jul 07 15:06:58.251038 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.251018    2556 scope.go:117] "RemoveContainer" containerID="689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a"
Jul 07 15:06:58.258659 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.258526    2556 scope.go:117] "RemoveContainer" containerID="617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1"
Jul 07 15:06:58.258820 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:06:58.258780    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1\": container with ID starting with 617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1 not found: ID does not exist" containerID="617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1"
Jul 07 15:06:58.258868 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.258822    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1"} err="failed to get container status \"617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1\": rpc error: code = NotFound desc = could not find container \"617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1\": container with ID starting with 617018d9af6cbfffd118cb7ce77915848fd52665bf65e398a36210bf0dc701c1 not found: ID does not exist"
Jul 07 15:06:58.258868 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.258838    2556 scope.go:117] "RemoveContainer" containerID="689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a"
Jul 07 15:06:58.259077 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:06:58.259052    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a\": container with ID starting with 689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a not found: ID does not exist" containerID="689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a"
Jul 07 15:06:58.259115 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.259082    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a"} err="failed to get container status \"689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a\": rpc error: code = NotFound desc = could not find container \"689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a\": container with ID starting with 689204231edf1555cdd8c14ae3087ca85ae5d3570e2993a9f93d4e267818a92a not found: ID does not exist"
Jul 07 15:06:58.267360 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.267338    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"]
Jul 07 15:06:58.269109 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:58.269090    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f85ce-predictor-6578cf6d78-c57pb"]
Jul 07 15:06:59.270198 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:06:59.270165    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" path="/var/lib/kubelet/pods/2448e304-2d49-4f37-bdf8-f5598d6e4698/volumes"
Jul 07 15:07:02.187975 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:02.187932    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:02.239763 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:02.239737    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:07:02.240280 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:02.240255    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 15:07:07.187413 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:07.187371    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:07.187816 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:07.187498    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:07:12.187311 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:12.187274    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:12.240975 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:12.240932    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 15:07:17.187004 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:17.186960    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:22.187339 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:22.187290    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:22.240744 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:22.240709    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 15:07:24.001487 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.001459    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:07:24.128272 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.128200    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls\") pod \"9cf7b141-0758-4ef9-85f0-a69102495a37\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") "
Jul 07 15:07:24.128403 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.128290    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf7b141-0758-4ef9-85f0-a69102495a37-openshift-service-ca-bundle\") pod \"9cf7b141-0758-4ef9-85f0-a69102495a37\" (UID: \"9cf7b141-0758-4ef9-85f0-a69102495a37\") "
Jul 07 15:07:24.128659 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.128627    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf7b141-0758-4ef9-85f0-a69102495a37-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9cf7b141-0758-4ef9-85f0-a69102495a37" (UID: "9cf7b141-0758-4ef9-85f0-a69102495a37"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:07:24.130186 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.130166    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9cf7b141-0758-4ef9-85f0-a69102495a37" (UID: "9cf7b141-0758-4ef9-85f0-a69102495a37"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:07:24.229438 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.229411    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf7b141-0758-4ef9-85f0-a69102495a37-openshift-service-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:07:24.229438 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.229435    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9cf7b141-0758-4ef9-85f0-a69102495a37-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:07:24.369728 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.369694    2556 generic.go:356] "Generic (PLEG): container finished" podID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerID="463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9" exitCode=0
Jul 07 15:07:24.369891 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.369732    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"
Jul 07 15:07:24.369891 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.369777    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" event={"ID":"9cf7b141-0758-4ef9-85f0-a69102495a37","Type":"ContainerDied","Data":"463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9"}
Jul 07 15:07:24.369891 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.369849    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847" event={"ID":"9cf7b141-0758-4ef9-85f0-a69102495a37","Type":"ContainerDied","Data":"e3b206e3f78b1bba08924dea7a318779736716417abbe11d83087bfba4a6c3c4"}
Jul 07 15:07:24.369891 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.369876    2556 scope.go:117] "RemoveContainer" containerID="463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9"
Jul 07 15:07:24.379170 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.379136    2556 scope.go:117] "RemoveContainer" containerID="463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9"
Jul 07 15:07:24.379427 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:07:24.379401    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9\": container with ID starting with 463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9 not found: ID does not exist" containerID="463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9"
Jul 07 15:07:24.379506 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.379434    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9"} err="failed to get container status \"463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9\": rpc error: code = NotFound desc = could not find container \"463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9\": container with ID starting with 463cbfd24b4aef8c253136e092971c9b4009f7fac5112dfc3e73966186320ba9 not found: ID does not exist"
Jul 07 15:07:24.392112 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.392086    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"]
Jul 07 15:07:24.397682 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:24.397662    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f85ce-7d456c44cd-fj847"]
Jul 07 15:07:25.270678 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:25.270645    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" path="/var/lib/kubelet/pods/9cf7b141-0758-4ef9-85f0-a69102495a37/volumes"
Jul 07 15:07:29.956484 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.956455    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"]
Jul 07 15:07:29.956890 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.956876    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce"
Jul 07 15:07:29.956944 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.956892    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce"
Jul 07 15:07:29.956944 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.956906    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kube-rbac-proxy"
Jul 07 15:07:29.956944 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.956911    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kube-rbac-proxy"
Jul 07 15:07:29.956944 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.956929    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container"
Jul 07 15:07:29.956944 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.956937    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container"
Jul 07 15:07:29.957097 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.957007    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kserve-container"
Jul 07 15:07:29.957097 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.957018    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf7b141-0758-4ef9-85f0-a69102495a37" containerName="switch-graph-f85ce"
Jul 07 15:07:29.957097 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.957027    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2448e304-2d49-4f37-bdf8-f5598d6e4698" containerName="kube-rbac-proxy"
Jul 07 15:07:29.961442 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.961422    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:29.964898 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.964863    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8996c-kube-rbac-proxy-sar-config\""
Jul 07 15:07:29.965457 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.965433    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8996c-predictor-serving-cert\""
Jul 07 15:07:29.976894 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:29.976875    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"]
Jul 07 15:07:30.072158 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.072132    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.072312 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.072287    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.072473 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.072350    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vrw7\" (UniqueName: \"kubernetes.io/projected/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-api-access-8vrw7\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.173077 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.173054    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.173228 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.173085    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vrw7\" (UniqueName: \"kubernetes.io/projected/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-api-access-8vrw7\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.173228 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.173116    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.173228 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:07:30.173181    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-8996c-predictor-serving-cert: secret "success-200-isvc-8996c-predictor-serving-cert" not found
Jul 07 15:07:30.173402 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:07:30.173253    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls podName:135de414-c12c-40d5-ba2a-1ba5614d3e18 nodeName:}" failed. No retries permitted until 2026-07-07 15:07:30.67323305 +0000 UTC m=+668.021715664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls") pod "success-200-isvc-8996c-predictor-bb4b8bbb-462gw" (UID: "135de414-c12c-40d5-ba2a-1ba5614d3e18") : secret "success-200-isvc-8996c-predictor-serving-cert" not found
Jul 07 15:07:30.173713 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.173696    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.183615 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.183591    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vrw7\" (UniqueName: \"kubernetes.io/projected/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-api-access-8vrw7\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.677979 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.677948    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.680352 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.680327    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls\") pod \"success-200-isvc-8996c-predictor-bb4b8bbb-462gw\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") " pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:30.875563 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:30.875533    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:31.004419 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.004392    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"]
Jul 07 15:07:31.006376 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:07:31.006353    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod135de414_c12c_40d5_ba2a_1ba5614d3e18.slice/crio-3b0d9d761a03af87a674f6f5fa0c3b231577f8a6ce87bddbf620e348b87b1714 WatchSource:0}: Error finding container 3b0d9d761a03af87a674f6f5fa0c3b231577f8a6ce87bddbf620e348b87b1714: Status 404 returned error can't find the container with id 3b0d9d761a03af87a674f6f5fa0c3b231577f8a6ce87bddbf620e348b87b1714
Jul 07 15:07:31.403848 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.403725    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" event={"ID":"135de414-c12c-40d5-ba2a-1ba5614d3e18","Type":"ContainerStarted","Data":"5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c"}
Jul 07 15:07:31.404004 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.403845    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" event={"ID":"135de414-c12c-40d5-ba2a-1ba5614d3e18","Type":"ContainerStarted","Data":"3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026"}
Jul 07 15:07:31.404004 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.403883    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:31.404004 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.403916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" event={"ID":"135de414-c12c-40d5-ba2a-1ba5614d3e18","Type":"ContainerStarted","Data":"3b0d9d761a03af87a674f6f5fa0c3b231577f8a6ce87bddbf620e348b87b1714"}
Jul 07 15:07:31.404004 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.403945    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:31.405762 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.405735    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 15:07:31.423957 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:31.423918    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podStartSLOduration=2.423903592 podStartE2EDuration="2.423903592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:31.423076289 +0000 UTC m=+668.771558919" watchObservedRunningTime="2026-07-07 15:07:31.423903592 +0000 UTC m=+668.772386223"
Jul 07 15:07:32.240469 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:32.240434    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 15:07:32.408204 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:32.408168    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 15:07:37.414451 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:37.414423    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:07:37.414876 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:37.414849    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 15:07:42.240975 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:42.240944    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:07:47.415607 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:47.415563    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 15:07:57.415904 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:07:57.415861    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 15:08:07.415311 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:08:07.415274    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 15:08:17.416517 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:08:17.416439    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:11:23.165879 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:11:23.165853    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:11:23.170865 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:11:23.170844    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:11:23.173358 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:11:23.173334    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:11:23.177925 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:11:23.177909    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:16:09.208764 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.208726    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"]
Jul 07 15:16:09.209409 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.209045    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kube-rbac-proxy" containerID="cri-o://9ceae840324d4568616c83d7cde38aa601635bd492424d279dc1130d77cd689d" gracePeriod=30
Jul 07 15:16:09.209409 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.208996    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container" containerID="cri-o://cc83f8c0f1a9869498c6e6c6bebdb36ceb86b9f50132ba150f52c5a8dab934ee" gracePeriod=30
Jul 07 15:16:09.212301 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.212278    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"]
Jul 07 15:16:09.216368 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.216284    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.220273 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.220251    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-542f9-predictor-serving-cert\""
Jul 07 15:16:09.220367 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.220252    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-542f9-kube-rbac-proxy-sar-config\""
Jul 07 15:16:09.230663 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.230640    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"]
Jul 07 15:16:09.288597 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.288574    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8014b647-4160-4398-b762-80112c6978fe-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.288704 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.288649    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.288704 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.288685    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpppr\" (UniqueName: \"kubernetes.io/projected/8014b647-4160-4398-b762-80112c6978fe-kube-api-access-bpppr\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.389393 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.389360    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8014b647-4160-4398-b762-80112c6978fe-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.389546 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.389425    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.389546 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.389465    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpppr\" (UniqueName: \"kubernetes.io/projected/8014b647-4160-4398-b762-80112c6978fe-kube-api-access-bpppr\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.389546 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:16:09.389533    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-542f9-predictor-serving-cert: secret "success-200-isvc-542f9-predictor-serving-cert" not found
Jul 07 15:16:09.389670 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:16:09.389611    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls podName:8014b647-4160-4398-b762-80112c6978fe nodeName:}" failed. No retries permitted until 2026-07-07 15:16:09.889590598 +0000 UTC m=+1187.238073212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls") pod "success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" (UID: "8014b647-4160-4398-b762-80112c6978fe") : secret "success-200-isvc-542f9-predictor-serving-cert" not found
Jul 07 15:16:09.390127 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.390107    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8014b647-4160-4398-b762-80112c6978fe-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.400162 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.400138    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpppr\" (UniqueName: \"kubernetes.io/projected/8014b647-4160-4398-b762-80112c6978fe-kube-api-access-bpppr\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.876947 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.876895    2556 generic.go:356] "Generic (PLEG): container finished" podID="df197fac-db99-4475-b945-3b9f0073656d" containerID="9ceae840324d4568616c83d7cde38aa601635bd492424d279dc1130d77cd689d" exitCode=2
Jul 07 15:16:09.877124 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.876968    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" event={"ID":"df197fac-db99-4475-b945-3b9f0073656d","Type":"ContainerDied","Data":"9ceae840324d4568616c83d7cde38aa601635bd492424d279dc1130d77cd689d"}
Jul 07 15:16:09.894351 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:09.894324    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:09.894468 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:16:09.894457    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-542f9-predictor-serving-cert: secret "success-200-isvc-542f9-predictor-serving-cert" not found
Jul 07 15:16:09.894529 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:16:09.894518    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls podName:8014b647-4160-4398-b762-80112c6978fe nodeName:}" failed. No retries permitted until 2026-07-07 15:16:10.894504736 +0000 UTC m=+1188.242987344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls") pod "success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" (UID: "8014b647-4160-4398-b762-80112c6978fe") : secret "success-200-isvc-542f9-predictor-serving-cert" not found
Jul 07 15:16:10.904523 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:10.904490    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:10.906854 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:10.906824    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls\") pod \"success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") " pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:11.039878 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.039850    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:11.170343 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.170316    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"]
Jul 07 15:16:11.172488 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:16:11.172463    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8014b647_4160_4398_b762_80112c6978fe.slice/crio-2c6865106a98987323de7b56445df5e71ca6c2dfdca69202cc3074dd6b09bfaf WatchSource:0}: Error finding container 2c6865106a98987323de7b56445df5e71ca6c2dfdca69202cc3074dd6b09bfaf: Status 404 returned error can't find the container with id 2c6865106a98987323de7b56445df5e71ca6c2dfdca69202cc3074dd6b09bfaf
Jul 07 15:16:11.891077 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.891043    2556 generic.go:356] "Generic (PLEG): container finished" podID="df197fac-db99-4475-b945-3b9f0073656d" containerID="cc83f8c0f1a9869498c6e6c6bebdb36ceb86b9f50132ba150f52c5a8dab934ee" exitCode=0
Jul 07 15:16:11.891228 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.891078    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" event={"ID":"df197fac-db99-4475-b945-3b9f0073656d","Type":"ContainerDied","Data":"cc83f8c0f1a9869498c6e6c6bebdb36ceb86b9f50132ba150f52c5a8dab934ee"}
Jul 07 15:16:11.893546 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.893518    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" event={"ID":"8014b647-4160-4398-b762-80112c6978fe","Type":"ContainerStarted","Data":"dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf"}
Jul 07 15:16:11.893672 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.893580    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" event={"ID":"8014b647-4160-4398-b762-80112c6978fe","Type":"ContainerStarted","Data":"30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11"}
Jul 07 15:16:11.893672 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.893613    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" event={"ID":"8014b647-4160-4398-b762-80112c6978fe","Type":"ContainerStarted","Data":"2c6865106a98987323de7b56445df5e71ca6c2dfdca69202cc3074dd6b09bfaf"}
Jul 07 15:16:11.893672 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.893650    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:11.893812 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.893767    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:11.895070 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.895048    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 15:16:11.914615 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.914567    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podStartSLOduration=2.914547531 podStartE2EDuration="2.914547531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:16:11.912102722 +0000 UTC m=+1189.260585354" watchObservedRunningTime="2026-07-07 15:16:11.914547531 +0000 UTC m=+1189.263030162"
Jul 07 15:16:11.958005 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:11.957988    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:16:12.012755 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.012733    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvn6s\" (UniqueName: \"kubernetes.io/projected/df197fac-db99-4475-b945-3b9f0073656d-kube-api-access-tvn6s\") pod \"df197fac-db99-4475-b945-3b9f0073656d\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") "
Jul 07 15:16:12.012880 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.012787    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df197fac-db99-4475-b945-3b9f0073656d-proxy-tls\") pod \"df197fac-db99-4475-b945-3b9f0073656d\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") "
Jul 07 15:16:12.012880 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.012856    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df197fac-db99-4475-b945-3b9f0073656d-kube-rbac-proxy-sar-config\") pod \"df197fac-db99-4475-b945-3b9f0073656d\" (UID: \"df197fac-db99-4475-b945-3b9f0073656d\") "
Jul 07 15:16:12.013208 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.013181    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df197fac-db99-4475-b945-3b9f0073656d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "df197fac-db99-4475-b945-3b9f0073656d" (UID: "df197fac-db99-4475-b945-3b9f0073656d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:16:12.014679 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.014652    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df197fac-db99-4475-b945-3b9f0073656d-kube-api-access-tvn6s" (OuterVolumeSpecName: "kube-api-access-tvn6s") pod "df197fac-db99-4475-b945-3b9f0073656d" (UID: "df197fac-db99-4475-b945-3b9f0073656d"). InnerVolumeSpecName "kube-api-access-tvn6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:16:12.014843 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.014825    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df197fac-db99-4475-b945-3b9f0073656d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "df197fac-db99-4475-b945-3b9f0073656d" (UID: "df197fac-db99-4475-b945-3b9f0073656d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:16:12.113422 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.113360    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvn6s\" (UniqueName: \"kubernetes.io/projected/df197fac-db99-4475-b945-3b9f0073656d-kube-api-access-tvn6s\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:16:12.113422 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.113390    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df197fac-db99-4475-b945-3b9f0073656d-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:16:12.113422 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.113399    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df197fac-db99-4475-b945-3b9f0073656d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:16:12.900418 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.900390    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"
Jul 07 15:16:12.900571 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.900431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs" event={"ID":"df197fac-db99-4475-b945-3b9f0073656d","Type":"ContainerDied","Data":"2bd3510040be4b1685e6c02be7481293f0aa126ec00d9a1a952df5e459e3a87d"}
Jul 07 15:16:12.900571 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.900503    2556 scope.go:117] "RemoveContainer" containerID="9ceae840324d4568616c83d7cde38aa601635bd492424d279dc1130d77cd689d"
Jul 07 15:16:12.900828 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.900779    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 15:16:12.909547 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.909531    2556 scope.go:117] "RemoveContainer" containerID="cc83f8c0f1a9869498c6e6c6bebdb36ceb86b9f50132ba150f52c5a8dab934ee"
Jul 07 15:16:12.923722 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.923702    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"]
Jul 07 15:16:12.927673 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:12.927653    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3c9c9-predictor-854b5487bb-zz8fs"]
Jul 07 15:16:13.269809 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:13.269769    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df197fac-db99-4475-b945-3b9f0073656d" path="/var/lib/kubelet/pods/df197fac-db99-4475-b945-3b9f0073656d/volumes"
Jul 07 15:16:17.905206 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:17.905180    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:16:17.905645 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:17.905567    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 15:16:23.199900 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:23.199868    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:16:23.204451 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:23.204426    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:16:23.205630 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:23.205607    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:16:23.210000 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:23.209980    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:16:27.905682 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:27.905640    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 15:16:37.905611 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:37.905573    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 15:16:44.899126 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.899089    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"]
Jul 07 15:16:44.899572 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.899299    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" containerID="cri-o://3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026" gracePeriod=30
Jul 07 15:16:44.899572 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.899348    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kube-rbac-proxy" containerID="cri-o://5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c" gracePeriod=30
Jul 07 15:16:44.924957 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.924928    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"]
Jul 07 15:16:44.925323 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.925310    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container"
Jul 07 15:16:44.925367 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.925324    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container"
Jul 07 15:16:44.925367 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.925333    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kube-rbac-proxy"
Jul 07 15:16:44.925367 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.925339    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kube-rbac-proxy"
Jul 07 15:16:44.925460 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.925396    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kube-rbac-proxy"
Jul 07 15:16:44.925460 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.925404    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="df197fac-db99-4475-b945-3b9f0073656d" containerName="kserve-container"
Jul 07 15:16:44.929835 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.929776    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:44.932368 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.932333    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e4605-kube-rbac-proxy-sar-config\""
Jul 07 15:16:44.932469 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.932429    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e4605-predictor-serving-cert\""
Jul 07 15:16:44.941866 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:44.941846    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"]
Jul 07 15:16:45.052256 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.052221    2556 generic.go:356] "Generic (PLEG): container finished" podID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerID="5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c" exitCode=2
Jul 07 15:16:45.052398 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.052288    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" event={"ID":"135de414-c12c-40d5-ba2a-1ba5614d3e18","Type":"ContainerDied","Data":"5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c"}
Jul 07 15:16:45.065370 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.065344    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb042c00-7d75-463c-baa9-1df984077702-proxy-tls\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.065510 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.065399    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jpl5\" (UniqueName: \"kubernetes.io/projected/eb042c00-7d75-463c-baa9-1df984077702-kube-api-access-8jpl5\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.065510 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.065456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb042c00-7d75-463c-baa9-1df984077702-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.166490 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.166451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb042c00-7d75-463c-baa9-1df984077702-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.166637 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.166501    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb042c00-7d75-463c-baa9-1df984077702-proxy-tls\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.166637 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.166549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jpl5\" (UniqueName: \"kubernetes.io/projected/eb042c00-7d75-463c-baa9-1df984077702-kube-api-access-8jpl5\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.167142 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.167120    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb042c00-7d75-463c-baa9-1df984077702-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.168747 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.168727    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb042c00-7d75-463c-baa9-1df984077702-proxy-tls\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.179764 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.179739    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jpl5\" (UniqueName: \"kubernetes.io/projected/eb042c00-7d75-463c-baa9-1df984077702-kube-api-access-8jpl5\") pod \"success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") " pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.241555 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.241524    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:45.377456 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:45.377430    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"]
Jul 07 15:16:45.378932 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:16:45.378906    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb042c00_7d75_463c_baa9_1df984077702.slice/crio-3afe978c366ad71a35e45f38239cc86aeacbeed53fc011e6b03019c25e87f011 WatchSource:0}: Error finding container 3afe978c366ad71a35e45f38239cc86aeacbeed53fc011e6b03019c25e87f011: Status 404 returned error can't find the container with id 3afe978c366ad71a35e45f38239cc86aeacbeed53fc011e6b03019c25e87f011
Jul 07 15:16:46.059118 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:46.059085    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" event={"ID":"eb042c00-7d75-463c-baa9-1df984077702","Type":"ContainerStarted","Data":"ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8"}
Jul 07 15:16:46.059515 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:46.059139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" event={"ID":"eb042c00-7d75-463c-baa9-1df984077702","Type":"ContainerStarted","Data":"559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216"}
Jul 07 15:16:46.059515 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:46.059161    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" event={"ID":"eb042c00-7d75-463c-baa9-1df984077702","Type":"ContainerStarted","Data":"3afe978c366ad71a35e45f38239cc86aeacbeed53fc011e6b03019c25e87f011"}
Jul 07 15:16:46.059515 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:46.059183    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:46.059515 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:46.059202    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:46.060776 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:46.060748    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 15:16:46.080560 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:46.080509    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podStartSLOduration=2.080493698 podStartE2EDuration="2.080493698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:16:46.078302058 +0000 UTC m=+1223.426784688" watchObservedRunningTime="2026-07-07 15:16:46.080493698 +0000 UTC m=+1223.428976385"
Jul 07 15:16:47.063580 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.063537    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 15:16:47.409443 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.409355    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused"
Jul 07 15:16:47.415487 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.415454    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 15:16:47.760113 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.760087    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:16:47.892491 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.892461    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vrw7\" (UniqueName: \"kubernetes.io/projected/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-api-access-8vrw7\") pod \"135de414-c12c-40d5-ba2a-1ba5614d3e18\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") "
Jul 07 15:16:47.892690 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.892550    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-rbac-proxy-sar-config\") pod \"135de414-c12c-40d5-ba2a-1ba5614d3e18\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") "
Jul 07 15:16:47.892690 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.892600    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls\") pod \"135de414-c12c-40d5-ba2a-1ba5614d3e18\" (UID: \"135de414-c12c-40d5-ba2a-1ba5614d3e18\") "
Jul 07 15:16:47.892955 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.892929    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "135de414-c12c-40d5-ba2a-1ba5614d3e18" (UID: "135de414-c12c-40d5-ba2a-1ba5614d3e18"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:16:47.894608 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.894583    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "135de414-c12c-40d5-ba2a-1ba5614d3e18" (UID: "135de414-c12c-40d5-ba2a-1ba5614d3e18"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:16:47.894608 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.894596    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-api-access-8vrw7" (OuterVolumeSpecName: "kube-api-access-8vrw7") pod "135de414-c12c-40d5-ba2a-1ba5614d3e18" (UID: "135de414-c12c-40d5-ba2a-1ba5614d3e18"). InnerVolumeSpecName "kube-api-access-8vrw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:16:47.906418 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.906392    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 15:16:47.994171 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.994113    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:16:47.994171 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.994135    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/135de414-c12c-40d5-ba2a-1ba5614d3e18-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:16:47.994171 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:47.994147    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vrw7\" (UniqueName: \"kubernetes.io/projected/135de414-c12c-40d5-ba2a-1ba5614d3e18-kube-api-access-8vrw7\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:16:48.070121 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.070093    2556 generic.go:356] "Generic (PLEG): container finished" podID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerID="3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026" exitCode=0
Jul 07 15:16:48.070450 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.070154    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"
Jul 07 15:16:48.070450 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.070175    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" event={"ID":"135de414-c12c-40d5-ba2a-1ba5614d3e18","Type":"ContainerDied","Data":"3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026"}
Jul 07 15:16:48.070450 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.070251    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw" event={"ID":"135de414-c12c-40d5-ba2a-1ba5614d3e18","Type":"ContainerDied","Data":"3b0d9d761a03af87a674f6f5fa0c3b231577f8a6ce87bddbf620e348b87b1714"}
Jul 07 15:16:48.070450 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.070280    2556 scope.go:117] "RemoveContainer" containerID="5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c"
Jul 07 15:16:48.079110 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.079092    2556 scope.go:117] "RemoveContainer" containerID="3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026"
Jul 07 15:16:48.086775 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.086758    2556 scope.go:117] "RemoveContainer" containerID="5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c"
Jul 07 15:16:48.087085 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:16:48.087064    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c\": container with ID starting with 5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c not found: ID does not exist" containerID="5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c"
Jul 07 15:16:48.087173 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.087096    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c"} err="failed to get container status \"5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c\": rpc error: code = NotFound desc = could not find container \"5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c\": container with ID starting with 5452d1e99154a388e4e44831f94933d869f08ea8eb58ff3341a18820f88fbf5c not found: ID does not exist"
Jul 07 15:16:48.087173 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.087121    2556 scope.go:117] "RemoveContainer" containerID="3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026"
Jul 07 15:16:48.087356 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:16:48.087336    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026\": container with ID starting with 3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026 not found: ID does not exist" containerID="3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026"
Jul 07 15:16:48.087399 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.087364    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026"} err="failed to get container status \"3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026\": rpc error: code = NotFound desc = could not find container \"3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026\": container with ID starting with 3c81595c9b97a7ced5db4f2146f1547a3fb45173eb252b626a604c5cf5eb3026 not found: ID does not exist"
Jul 07 15:16:48.094425 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.094293    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"]
Jul 07 15:16:48.096757 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:48.096737    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8996c-predictor-bb4b8bbb-462gw"]
Jul 07 15:16:49.270718 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:49.270686    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" path="/var/lib/kubelet/pods/135de414-c12c-40d5-ba2a-1ba5614d3e18/volumes"
Jul 07 15:16:52.069591 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:52.069563    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:16:52.070052 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:52.070022    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 15:16:57.906492 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:16:57.906465    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:17:02.071028 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:02.070986    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 15:17:12.070008 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:12.069966    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 15:17:19.575891 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.575861    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"]
Jul 07 15:17:19.576268 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.576231    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kube-rbac-proxy"
Jul 07 15:17:19.576268 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.576242    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kube-rbac-proxy"
Jul 07 15:17:19.576268 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.576267    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container"
Jul 07 15:17:19.576372 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.576272    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container"
Jul 07 15:17:19.576372 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.576324    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kserve-container"
Jul 07 15:17:19.576372 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.576332    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="135de414-c12c-40d5-ba2a-1ba5614d3e18" containerName="kube-rbac-proxy"
Jul 07 15:17:19.579592 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.579572    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.586471 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.586451    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5aeac-predictor-serving-cert\""
Jul 07 15:17:19.587285 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.587268    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5aeac-kube-rbac-proxy-sar-config\""
Jul 07 15:17:19.607864 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.607844    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"]
Jul 07 15:17:19.649673 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.649647    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4b8ce98-14b0-4949-8976-836e72025179-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.649925 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.649703    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4b8ce98-14b0-4949-8976-836e72025179-proxy-tls\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.649925 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.649771    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrwt\" (UniqueName: \"kubernetes.io/projected/f4b8ce98-14b0-4949-8976-836e72025179-kube-api-access-jqrwt\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.653300 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.653256    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"]
Jul 07 15:17:19.653513 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.653485    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container" containerID="cri-o://30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11" gracePeriod=30
Jul 07 15:17:19.653625 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.653527    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kube-rbac-proxy" containerID="cri-o://dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf" gracePeriod=30
Jul 07 15:17:19.750981 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.750957    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4b8ce98-14b0-4949-8976-836e72025179-proxy-tls\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.751114 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.751002    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrwt\" (UniqueName: \"kubernetes.io/projected/f4b8ce98-14b0-4949-8976-836e72025179-kube-api-access-jqrwt\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.751114 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.751057    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4b8ce98-14b0-4949-8976-836e72025179-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.751592 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.751573    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4b8ce98-14b0-4949-8976-836e72025179-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.753363 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.753341    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4b8ce98-14b0-4949-8976-836e72025179-proxy-tls\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.764108 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.764086    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrwt\" (UniqueName: \"kubernetes.io/projected/f4b8ce98-14b0-4949-8976-836e72025179-kube-api-access-jqrwt\") pod \"success-200-isvc-5aeac-predictor-74845fc44b-94nkv\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") " pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:19.889234 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:19.889167    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:20.025224 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.025188    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"]
Jul 07 15:17:20.027182 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:17:20.027154    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b8ce98_14b0_4949_8976_836e72025179.slice/crio-43e4a634d06af36fe3f3e0935d14570023b13d65f0ca084a08330b3234e267f1 WatchSource:0}: Error finding container 43e4a634d06af36fe3f3e0935d14570023b13d65f0ca084a08330b3234e267f1: Status 404 returned error can't find the container with id 43e4a634d06af36fe3f3e0935d14570023b13d65f0ca084a08330b3234e267f1
Jul 07 15:17:20.223476 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.223452    2556 generic.go:356] "Generic (PLEG): container finished" podID="8014b647-4160-4398-b762-80112c6978fe" containerID="dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf" exitCode=2
Jul 07 15:17:20.223631 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.223525    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" event={"ID":"8014b647-4160-4398-b762-80112c6978fe","Type":"ContainerDied","Data":"dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf"}
Jul 07 15:17:20.225174 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.225155    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" event={"ID":"f4b8ce98-14b0-4949-8976-836e72025179","Type":"ContainerStarted","Data":"29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835"}
Jul 07 15:17:20.225311 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.225210    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" event={"ID":"f4b8ce98-14b0-4949-8976-836e72025179","Type":"ContainerStarted","Data":"b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375"}
Jul 07 15:17:20.225311 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.225260    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" event={"ID":"f4b8ce98-14b0-4949-8976-836e72025179","Type":"ContainerStarted","Data":"43e4a634d06af36fe3f3e0935d14570023b13d65f0ca084a08330b3234e267f1"}
Jul 07 15:17:20.225412 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.225395    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:20.246269 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:20.246232    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podStartSLOduration=1.246219057 podStartE2EDuration="1.246219057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:17:20.244007769 +0000 UTC m=+1257.592490431" watchObservedRunningTime="2026-07-07 15:17:20.246219057 +0000 UTC m=+1257.594701688"
Jul 07 15:17:21.230140 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:21.230098    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:21.231622 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:21.231593    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 07 15:17:22.070388 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.070344    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 15:17:22.235339 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.235301    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 07 15:17:22.515483 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.515461    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:17:22.573687 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.573661    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpppr\" (UniqueName: \"kubernetes.io/projected/8014b647-4160-4398-b762-80112c6978fe-kube-api-access-bpppr\") pod \"8014b647-4160-4398-b762-80112c6978fe\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") "
Jul 07 15:17:22.573789 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.573741    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls\") pod \"8014b647-4160-4398-b762-80112c6978fe\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") "
Jul 07 15:17:22.573789 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.573769    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8014b647-4160-4398-b762-80112c6978fe-kube-rbac-proxy-sar-config\") pod \"8014b647-4160-4398-b762-80112c6978fe\" (UID: \"8014b647-4160-4398-b762-80112c6978fe\") "
Jul 07 15:17:22.574215 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.574189    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8014b647-4160-4398-b762-80112c6978fe-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8014b647-4160-4398-b762-80112c6978fe" (UID: "8014b647-4160-4398-b762-80112c6978fe"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:17:22.576286 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.576262    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8014b647-4160-4398-b762-80112c6978fe" (UID: "8014b647-4160-4398-b762-80112c6978fe"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:17:22.576386 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.576370    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8014b647-4160-4398-b762-80112c6978fe-kube-api-access-bpppr" (OuterVolumeSpecName: "kube-api-access-bpppr") pod "8014b647-4160-4398-b762-80112c6978fe" (UID: "8014b647-4160-4398-b762-80112c6978fe"). InnerVolumeSpecName "kube-api-access-bpppr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:17:22.674603 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.674570    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8014b647-4160-4398-b762-80112c6978fe-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:17:22.674603 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.674597    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8014b647-4160-4398-b762-80112c6978fe-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:17:22.674603 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:22.674607    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpppr\" (UniqueName: \"kubernetes.io/projected/8014b647-4160-4398-b762-80112c6978fe-kube-api-access-bpppr\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:17:23.241472 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.241438    2556 generic.go:356] "Generic (PLEG): container finished" podID="8014b647-4160-4398-b762-80112c6978fe" containerID="30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11" exitCode=0
Jul 07 15:17:23.241956 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.241484    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" event={"ID":"8014b647-4160-4398-b762-80112c6978fe","Type":"ContainerDied","Data":"30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11"}
Jul 07 15:17:23.241956 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.241505    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"
Jul 07 15:17:23.241956 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.241569    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc" event={"ID":"8014b647-4160-4398-b762-80112c6978fe","Type":"ContainerDied","Data":"2c6865106a98987323de7b56445df5e71ca6c2dfdca69202cc3074dd6b09bfaf"}
Jul 07 15:17:23.241956 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.241596    2556 scope.go:117] "RemoveContainer" containerID="dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf"
Jul 07 15:17:23.254987 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.254973    2556 scope.go:117] "RemoveContainer" containerID="30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11"
Jul 07 15:17:23.256174 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:17:23.256148    2556 kuberuntime_gc.go:392] "Failed to remove container log dead symlink" err="remove /var/log/containers/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc_kserve-ci-e2e-test_kserve-container-30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11.log: no such file or directory" path="/var/log/containers/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc_kserve-ci-e2e-test_kserve-container-30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11.log"
Jul 07 15:17:23.262271 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.262256    2556 scope.go:117] "RemoveContainer" containerID="dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf"
Jul 07 15:17:23.262492 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:17:23.262474    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf\": container with ID starting with dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf not found: ID does not exist" containerID="dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf"
Jul 07 15:17:23.262545 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.262498    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf"} err="failed to get container status \"dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf\": rpc error: code = NotFound desc = could not find container \"dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf\": container with ID starting with dbfc8163e6cacbe2fe4a9649b44273f0e4492be961d54b30fac4c0ed2aab4dcf not found: ID does not exist"
Jul 07 15:17:23.262545 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.262512    2556 scope.go:117] "RemoveContainer" containerID="30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11"
Jul 07 15:17:23.262765 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:17:23.262739    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11\": container with ID starting with 30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11 not found: ID does not exist" containerID="30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11"
Jul 07 15:17:23.262882 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.262776    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11"} err="failed to get container status \"30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11\": rpc error: code = NotFound desc = could not find container \"30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11\": container with ID starting with 30b8ae5f9d8dc6025614c5863dad745b8a9d335fe9dabce25b020a3d76de5c11 not found: ID does not exist"
Jul 07 15:17:23.271928 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.271909    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"]
Jul 07 15:17:23.272013 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:23.271940    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-542f9-predictor-67b4c56d5b-4mbqc"]
Jul 07 15:17:25.269415 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:25.269385    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8014b647-4160-4398-b762-80112c6978fe" path="/var/lib/kubelet/pods/8014b647-4160-4398-b762-80112c6978fe/volumes"
Jul 07 15:17:27.239307 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:27.239281    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:17:27.239765 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:27.239737    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 07 15:17:32.071665 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:32.071636    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:17:37.239963 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:37.239925    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 07 15:17:47.239973 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:47.239931    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 07 15:17:55.160584 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.160551    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"]
Jul 07 15:17:55.161117 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.160804    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container" containerID="cri-o://559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216" gracePeriod=30
Jul 07 15:17:55.161117 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.160827    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kube-rbac-proxy" containerID="cri-o://ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8" gracePeriod=30
Jul 07 15:17:55.167207 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.167184    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"]
Jul 07 15:17:55.167676 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.167641    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kube-rbac-proxy"
Jul 07 15:17:55.167676 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.167667    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kube-rbac-proxy"
Jul 07 15:17:55.167887 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.167691    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container"
Jul 07 15:17:55.167887 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.167699    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container"
Jul 07 15:17:55.167887 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.167847    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kserve-container"
Jul 07 15:17:55.167887 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.167861    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8014b647-4160-4398-b762-80112c6978fe" containerName="kube-rbac-proxy"
Jul 07 15:17:55.171156 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.171139    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.173661 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.173644    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-a9c06-predictor-serving-cert\""
Jul 07 15:17:55.174075 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.174030    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-a9c06-kube-rbac-proxy-sar-config\""
Jul 07 15:17:55.183658 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.183636    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"]
Jul 07 15:17:55.342402 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.342374    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhfbf\" (UniqueName: \"kubernetes.io/projected/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-api-access-xhfbf\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.342562 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.342409    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-proxy-tls\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.342562 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.342445    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.389298 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.389263    2556 generic.go:356] "Generic (PLEG): container finished" podID="eb042c00-7d75-463c-baa9-1df984077702" containerID="ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8" exitCode=2
Jul 07 15:17:55.389447 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.389336    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" event={"ID":"eb042c00-7d75-463c-baa9-1df984077702","Type":"ContainerDied","Data":"ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8"}
Jul 07 15:17:55.443729 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.443657    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.443908 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.443787    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhfbf\" (UniqueName: \"kubernetes.io/projected/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-api-access-xhfbf\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.443908 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.443838    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-proxy-tls\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.444313 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.444292    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.446205 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.446183    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-proxy-tls\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.452347 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.452324    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhfbf\" (UniqueName: \"kubernetes.io/projected/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-api-access-xhfbf\") pod \"success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") " pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.486254 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.486227    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:55.625995 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:55.625939    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"]
Jul 07 15:17:55.628255 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:17:55.628231    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fca471a_5cc7_4a75_aa8b_5be08533e7ea.slice/crio-70fd7ba706ad038115fffe2b975e68cf5b38733fc3358edac4501adc50d638ad WatchSource:0}: Error finding container 70fd7ba706ad038115fffe2b975e68cf5b38733fc3358edac4501adc50d638ad: Status 404 returned error can't find the container with id 70fd7ba706ad038115fffe2b975e68cf5b38733fc3358edac4501adc50d638ad
Jul 07 15:17:56.395985 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:56.395945    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" event={"ID":"6fca471a-5cc7-4a75-aa8b-5be08533e7ea","Type":"ContainerStarted","Data":"90106e952f7e2c34fcb62069dabc259f2df214db793ff63cfdf3dd77c4728c16"}
Jul 07 15:17:56.396386 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:56.396010    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" event={"ID":"6fca471a-5cc7-4a75-aa8b-5be08533e7ea","Type":"ContainerStarted","Data":"123f68cc82f6ad1b68fbc20b85b9d7ff18fec7243af5c603825934fd0eb71d19"}
Jul 07 15:17:56.396386 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:56.396043    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" event={"ID":"6fca471a-5cc7-4a75-aa8b-5be08533e7ea","Type":"ContainerStarted","Data":"70fd7ba706ad038115fffe2b975e68cf5b38733fc3358edac4501adc50d638ad"}
Jul 07 15:17:56.396386 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:56.396074    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:56.396386 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:56.396104    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:17:56.397616 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:56.397592    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 07 15:17:56.416760 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:56.416712    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podStartSLOduration=1.416697634 podStartE2EDuration="1.416697634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:17:56.414057841 +0000 UTC m=+1293.762540470" watchObservedRunningTime="2026-07-07 15:17:56.416697634 +0000 UTC m=+1293.765180264"
Jul 07 15:17:57.064231 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:57.064191    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.41:8643/healthz\": dial tcp 10.132.0.41:8643: connect: connection refused"
Jul 07 15:17:57.240649 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:57.240611    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 07 15:17:57.401006 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:57.400910    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 07 15:17:58.715781 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.715761    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:17:58.876551 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.876480    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb042c00-7d75-463c-baa9-1df984077702-proxy-tls\") pod \"eb042c00-7d75-463c-baa9-1df984077702\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") "
Jul 07 15:17:58.876680 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.876558    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb042c00-7d75-463c-baa9-1df984077702-kube-rbac-proxy-sar-config\") pod \"eb042c00-7d75-463c-baa9-1df984077702\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") "
Jul 07 15:17:58.876680 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.876625    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jpl5\" (UniqueName: \"kubernetes.io/projected/eb042c00-7d75-463c-baa9-1df984077702-kube-api-access-8jpl5\") pod \"eb042c00-7d75-463c-baa9-1df984077702\" (UID: \"eb042c00-7d75-463c-baa9-1df984077702\") "
Jul 07 15:17:58.876941 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.876918    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb042c00-7d75-463c-baa9-1df984077702-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "eb042c00-7d75-463c-baa9-1df984077702" (UID: "eb042c00-7d75-463c-baa9-1df984077702"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:17:58.878708 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.878676    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb042c00-7d75-463c-baa9-1df984077702-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "eb042c00-7d75-463c-baa9-1df984077702" (UID: "eb042c00-7d75-463c-baa9-1df984077702"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:17:58.878835 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.878769    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb042c00-7d75-463c-baa9-1df984077702-kube-api-access-8jpl5" (OuterVolumeSpecName: "kube-api-access-8jpl5") pod "eb042c00-7d75-463c-baa9-1df984077702" (UID: "eb042c00-7d75-463c-baa9-1df984077702"). InnerVolumeSpecName "kube-api-access-8jpl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:17:58.978329 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.978300    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb042c00-7d75-463c-baa9-1df984077702-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:17:58.978329 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.978325    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jpl5\" (UniqueName: \"kubernetes.io/projected/eb042c00-7d75-463c-baa9-1df984077702-kube-api-access-8jpl5\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:17:58.978495 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:58.978337    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb042c00-7d75-463c-baa9-1df984077702-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:17:59.411649 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.411621    2556 generic.go:356] "Generic (PLEG): container finished" podID="eb042c00-7d75-463c-baa9-1df984077702" containerID="559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216" exitCode=0
Jul 07 15:17:59.411787 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.411663    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"
Jul 07 15:17:59.411787 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.411688    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" event={"ID":"eb042c00-7d75-463c-baa9-1df984077702","Type":"ContainerDied","Data":"559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216"}
Jul 07 15:17:59.411787 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.411745    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw" event={"ID":"eb042c00-7d75-463c-baa9-1df984077702","Type":"ContainerDied","Data":"3afe978c366ad71a35e45f38239cc86aeacbeed53fc011e6b03019c25e87f011"}
Jul 07 15:17:59.411787 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.411771    2556 scope.go:117] "RemoveContainer" containerID="ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8"
Jul 07 15:17:59.420147 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.420129    2556 scope.go:117] "RemoveContainer" containerID="559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216"
Jul 07 15:17:59.427389 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.427372    2556 scope.go:117] "RemoveContainer" containerID="ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8"
Jul 07 15:17:59.427637 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:17:59.427619    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8\": container with ID starting with ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8 not found: ID does not exist" containerID="ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8"
Jul 07 15:17:59.427681 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.427645    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8"} err="failed to get container status \"ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8\": rpc error: code = NotFound desc = could not find container \"ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8\": container with ID starting with ba16259f6bf565b2fc52a780b76ebbfcff06404e0cb1ccaa3550f34af38bfef8 not found: ID does not exist"
Jul 07 15:17:59.427681 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.427663    2556 scope.go:117] "RemoveContainer" containerID="559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216"
Jul 07 15:17:59.427918 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:17:59.427892    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216\": container with ID starting with 559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216 not found: ID does not exist" containerID="559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216"
Jul 07 15:17:59.427974 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.427925    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216"} err="failed to get container status \"559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216\": rpc error: code = NotFound desc = could not find container \"559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216\": container with ID starting with 559dbc5320a7ee2370612dc5f63c0dc52710376a8f800943a9e0d49a57a53216 not found: ID does not exist"
Jul 07 15:17:59.433025 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.433005    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"]
Jul 07 15:17:59.441185 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:17:59.441165    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e4605-predictor-5dcbcbf488-2vwnw"]
Jul 07 15:18:01.269669 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:01.269639    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb042c00-7d75-463c-baa9-1df984077702" path="/var/lib/kubelet/pods/eb042c00-7d75-463c-baa9-1df984077702/volumes"
Jul 07 15:18:02.406093 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:02.406063    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:18:02.406536 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:02.406509    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 07 15:18:07.240951 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:07.240925    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:18:12.407214 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:12.407175    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 07 15:18:22.407215 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:22.407176    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 07 15:18:32.406521 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:32.406481    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 07 15:18:42.406951 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:18:42.406919    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:21:23.239017 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:21:23.238981    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:21:23.239692 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:21:23.239670    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:21:23.244061 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:21:23.244036    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:21:23.244944 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:21:23.244925    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:26:23.275472 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:23.275445    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:26:23.276431 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:23.276407    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:26:23.280481 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:23.280451    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:26:23.281353 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:23.281336    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:26:34.684074 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.684040    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"]
Jul 07 15:26:34.684480 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.684337    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" containerID="cri-o://b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375" gracePeriod=30
Jul 07 15:26:34.684480 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.684337    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kube-rbac-proxy" containerID="cri-o://29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835" gracePeriod=30
Jul 07 15:26:34.687524 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.687497    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"]
Jul 07 15:26:34.688140 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.688118    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container"
Jul 07 15:26:34.688140 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.688143    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container"
Jul 07 15:26:34.688283 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.688176    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kube-rbac-proxy"
Jul 07 15:26:34.688283 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.688186    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kube-rbac-proxy"
Jul 07 15:26:34.688353 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.688285    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kube-rbac-proxy"
Jul 07 15:26:34.688353 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.688299    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb042c00-7d75-463c-baa9-1df984077702" containerName="kserve-container"
Jul 07 15:26:34.691786 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.691767    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.694347 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.694325    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-761e8-predictor-serving-cert\""
Jul 07 15:26:34.694455 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.694348    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-761e8-kube-rbac-proxy-sar-config\""
Jul 07 15:26:34.705355 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.705331    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"]
Jul 07 15:26:34.806160 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.806130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-proxy-tls\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.806299 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.806213    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb9s2\" (UniqueName: \"kubernetes.io/projected/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-api-access-fb9s2\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.806299 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.806253    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.880905 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.880871    2556 generic.go:356] "Generic (PLEG): container finished" podID="f4b8ce98-14b0-4949-8976-836e72025179" containerID="29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835" exitCode=2
Jul 07 15:26:34.881068 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.880950    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" event={"ID":"f4b8ce98-14b0-4949-8976-836e72025179","Type":"ContainerDied","Data":"29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835"}
Jul 07 15:26:34.907597 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.907569    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-proxy-tls\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.907746 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.907616    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb9s2\" (UniqueName: \"kubernetes.io/projected/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-api-access-fb9s2\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.907746 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.907642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.908424 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.908401    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.909988 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.909968    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-proxy-tls\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:34.919328 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:34.919300    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb9s2\" (UniqueName: \"kubernetes.io/projected/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-api-access-fb9s2\") pod \"success-200-isvc-761e8-predictor-6d97467478-f9qnw\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") " pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:35.010282 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.010212    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:35.149177 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.149156    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"]
Jul 07 15:26:35.151367 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:26:35.151336    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ee6064_4c5d_411b_b0e5_57823a6a9fa9.slice/crio-ecb4b3c759025f076cbe9d9128a11a319cf96d74101a0e751260d8c7d393d5c0 WatchSource:0}: Error finding container ecb4b3c759025f076cbe9d9128a11a319cf96d74101a0e751260d8c7d393d5c0: Status 404 returned error can't find the container with id ecb4b3c759025f076cbe9d9128a11a319cf96d74101a0e751260d8c7d393d5c0
Jul 07 15:26:35.887947 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.887912    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" event={"ID":"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9","Type":"ContainerStarted","Data":"4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8"}
Jul 07 15:26:35.888371 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.888017    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:35.888371 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.888037    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" event={"ID":"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9","Type":"ContainerStarted","Data":"3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17"}
Jul 07 15:26:35.888371 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.888064    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:35.888371 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.888086    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" event={"ID":"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9","Type":"ContainerStarted","Data":"ecb4b3c759025f076cbe9d9128a11a319cf96d74101a0e751260d8c7d393d5c0"}
Jul 07 15:26:35.889435 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.889410    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 07 15:26:35.913761 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:35.913720    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podStartSLOduration=1.913706637 podStartE2EDuration="1.913706637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:35.910146682 +0000 UTC m=+1813.258629331" watchObservedRunningTime="2026-07-07 15:26:35.913706637 +0000 UTC m=+1813.262189396"
Jul 07 15:26:36.893683 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:36.893641    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 07 15:26:37.236283 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.236243    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.42:8643/healthz\": dial tcp 10.132.0.42:8643: connect: connection refused"
Jul 07 15:26:37.240591 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.240566    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 07 15:26:37.444024 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.444002    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:26:37.527357 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.527303    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4b8ce98-14b0-4949-8976-836e72025179-kube-rbac-proxy-sar-config\") pod \"f4b8ce98-14b0-4949-8976-836e72025179\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") "
Jul 07 15:26:37.527476 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.527390    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4b8ce98-14b0-4949-8976-836e72025179-proxy-tls\") pod \"f4b8ce98-14b0-4949-8976-836e72025179\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") "
Jul 07 15:26:37.527476 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.527411    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqrwt\" (UniqueName: \"kubernetes.io/projected/f4b8ce98-14b0-4949-8976-836e72025179-kube-api-access-jqrwt\") pod \"f4b8ce98-14b0-4949-8976-836e72025179\" (UID: \"f4b8ce98-14b0-4949-8976-836e72025179\") "
Jul 07 15:26:37.527650 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.527623    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4b8ce98-14b0-4949-8976-836e72025179-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f4b8ce98-14b0-4949-8976-836e72025179" (UID: "f4b8ce98-14b0-4949-8976-836e72025179"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:26:37.529414 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.529387    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4b8ce98-14b0-4949-8976-836e72025179-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f4b8ce98-14b0-4949-8976-836e72025179" (UID: "f4b8ce98-14b0-4949-8976-836e72025179"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:26:37.529519 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.529444    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b8ce98-14b0-4949-8976-836e72025179-kube-api-access-jqrwt" (OuterVolumeSpecName: "kube-api-access-jqrwt") pod "f4b8ce98-14b0-4949-8976-836e72025179" (UID: "f4b8ce98-14b0-4949-8976-836e72025179"). InnerVolumeSpecName "kube-api-access-jqrwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:37.628690 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.628669    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4b8ce98-14b0-4949-8976-836e72025179-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:26:37.628690 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.628689    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4b8ce98-14b0-4949-8976-836e72025179-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:26:37.628844 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.628699    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqrwt\" (UniqueName: \"kubernetes.io/projected/f4b8ce98-14b0-4949-8976-836e72025179-kube-api-access-jqrwt\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:26:37.899450 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.899375    2556 generic.go:356] "Generic (PLEG): container finished" podID="f4b8ce98-14b0-4949-8976-836e72025179" containerID="b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375" exitCode=0
Jul 07 15:26:37.899450 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.899427    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"
Jul 07 15:26:37.899884 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.899457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" event={"ID":"f4b8ce98-14b0-4949-8976-836e72025179","Type":"ContainerDied","Data":"b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375"}
Jul 07 15:26:37.899884 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.899513    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv" event={"ID":"f4b8ce98-14b0-4949-8976-836e72025179","Type":"ContainerDied","Data":"43e4a634d06af36fe3f3e0935d14570023b13d65f0ca084a08330b3234e267f1"}
Jul 07 15:26:37.899884 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.899538    2556 scope.go:117] "RemoveContainer" containerID="29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835"
Jul 07 15:26:37.909550 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.909532    2556 scope.go:117] "RemoveContainer" containerID="b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375"
Jul 07 15:26:37.916892 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.916873    2556 scope.go:117] "RemoveContainer" containerID="29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835"
Jul 07 15:26:37.917137 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:26:37.917122    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835\": container with ID starting with 29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835 not found: ID does not exist" containerID="29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835"
Jul 07 15:26:37.917186 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.917146    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835"} err="failed to get container status \"29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835\": rpc error: code = NotFound desc = could not find container \"29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835\": container with ID starting with 29a2e91764b935edb9ae4e8335641aa4b0bdaeaad85bfb1721b310b026b1e835 not found: ID does not exist"
Jul 07 15:26:37.917186 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.917162    2556 scope.go:117] "RemoveContainer" containerID="b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375"
Jul 07 15:26:37.917377 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:26:37.917361    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375\": container with ID starting with b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375 not found: ID does not exist" containerID="b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375"
Jul 07 15:26:37.917414 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.917381    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375"} err="failed to get container status \"b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375\": rpc error: code = NotFound desc = could not find container \"b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375\": container with ID starting with b25bba3e3191384ae1aaf3934c475500dbbbbb90d981d40682a6b305ca5f6375 not found: ID does not exist"
Jul 07 15:26:37.934267 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.934244    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"]
Jul 07 15:26:37.939847 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:37.939831    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5aeac-predictor-74845fc44b-94nkv"]
Jul 07 15:26:39.269990 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:39.269959    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b8ce98-14b0-4949-8976-836e72025179" path="/var/lib/kubelet/pods/f4b8ce98-14b0-4949-8976-836e72025179/volumes"
Jul 07 15:26:41.897930 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:41.897902    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:26:41.898331 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:41.898308    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 07 15:26:51.899022 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:26:51.898984    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 07 15:27:01.898564 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:01.898521    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 07 15:27:10.168252 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.168219    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"]
Jul 07 15:27:10.168810 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.168777    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kube-rbac-proxy"
Jul 07 15:27:10.168907 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.168813    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kube-rbac-proxy"
Jul 07 15:27:10.168907 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.168842    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container"
Jul 07 15:27:10.168907 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.168851    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container"
Jul 07 15:27:10.169060 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.168956    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kserve-container"
Jul 07 15:27:10.169060 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.168973    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4b8ce98-14b0-4949-8976-836e72025179" containerName="kube-rbac-proxy"
Jul 07 15:27:10.172237 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.172219    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.175564 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.175546    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c3e9b-predictor-serving-cert\""
Jul 07 15:27:10.176085 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.176066    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c3e9b-kube-rbac-proxy-sar-config\""
Jul 07 15:27:10.190907 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.190887    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"]
Jul 07 15:27:10.258376 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.258344    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"]
Jul 07 15:27:10.258625 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.258597    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" containerID="cri-o://123f68cc82f6ad1b68fbc20b85b9d7ff18fec7243af5c603825934fd0eb71d19" gracePeriod=30
Jul 07 15:27:10.258754 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.258643    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kube-rbac-proxy" containerID="cri-o://90106e952f7e2c34fcb62069dabc259f2df214db793ff63cfdf3dd77c4728c16" gracePeriod=30
Jul 07 15:27:10.271427 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.271400    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4m79\" (UniqueName: \"kubernetes.io/projected/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-api-access-j4m79\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.271538 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.271482    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-proxy-tls\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.271602 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.271549    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.372846 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.372824    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-proxy-tls\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.372949 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.372895    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.372997 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.372945    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4m79\" (UniqueName: \"kubernetes.io/projected/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-api-access-j4m79\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.373572 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.373549    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.375203 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.375178    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-proxy-tls\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.395353 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.395328    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4m79\" (UniqueName: \"kubernetes.io/projected/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-api-access-j4m79\") pod \"success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.483897 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.483833    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:10.630899 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:10.630875    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"]
Jul 07 15:27:10.632394 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:27:10.632367    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbac4cca2_eea1_4e92_a48c_d8ba22d11f0f.slice/crio-4e02174a32ff0727c75c5be57ed02efbdc446af03ba7ad6313b44fd4d5d1b1af WatchSource:0}: Error finding container 4e02174a32ff0727c75c5be57ed02efbdc446af03ba7ad6313b44fd4d5d1b1af: Status 404 returned error can't find the container with id 4e02174a32ff0727c75c5be57ed02efbdc446af03ba7ad6313b44fd4d5d1b1af
Jul 07 15:27:11.059171 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.059083    2556 generic.go:356] "Generic (PLEG): container finished" podID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerID="90106e952f7e2c34fcb62069dabc259f2df214db793ff63cfdf3dd77c4728c16" exitCode=2
Jul 07 15:27:11.059171 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.059155    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" event={"ID":"6fca471a-5cc7-4a75-aa8b-5be08533e7ea","Type":"ContainerDied","Data":"90106e952f7e2c34fcb62069dabc259f2df214db793ff63cfdf3dd77c4728c16"}
Jul 07 15:27:11.060888 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.060855    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" event={"ID":"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f","Type":"ContainerStarted","Data":"3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7"}
Jul 07 15:27:11.061050 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.060906    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" event={"ID":"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f","Type":"ContainerStarted","Data":"de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8"}
Jul 07 15:27:11.061050 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.060928    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" event={"ID":"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f","Type":"ContainerStarted","Data":"4e02174a32ff0727c75c5be57ed02efbdc446af03ba7ad6313b44fd4d5d1b1af"}
Jul 07 15:27:11.061050 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.061011    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:11.095775 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.095732    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podStartSLOduration=1.095717616 podStartE2EDuration="1.095717616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:27:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:27:11.094680643 +0000 UTC m=+1848.443163275" watchObservedRunningTime="2026-07-07 15:27:11.095717616 +0000 UTC m=+1848.444200321"
Jul 07 15:27:11.898315 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:11.898274    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 07 15:27:12.066304 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:12.066270    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:12.067584 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:12.067559    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 07 15:27:12.401604 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:12.401559    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.43:8643/healthz\": dial tcp 10.132.0.43:8643: connect: connection refused"
Jul 07 15:27:12.406938 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:12.406903    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 07 15:27:13.074309 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.074267    2556 generic.go:356] "Generic (PLEG): container finished" podID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerID="123f68cc82f6ad1b68fbc20b85b9d7ff18fec7243af5c603825934fd0eb71d19" exitCode=0
Jul 07 15:27:13.074734 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.074338    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" event={"ID":"6fca471a-5cc7-4a75-aa8b-5be08533e7ea","Type":"ContainerDied","Data":"123f68cc82f6ad1b68fbc20b85b9d7ff18fec7243af5c603825934fd0eb71d19"}
Jul 07 15:27:13.074823 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.074775    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 07 15:27:13.113285 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.113266    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:27:13.197696 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.197670    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-rbac-proxy-sar-config\") pod \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") "
Jul 07 15:27:13.197827 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.197745    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhfbf\" (UniqueName: \"kubernetes.io/projected/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-api-access-xhfbf\") pod \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") "
Jul 07 15:27:13.197827 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.197808    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-proxy-tls\") pod \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\" (UID: \"6fca471a-5cc7-4a75-aa8b-5be08533e7ea\") "
Jul 07 15:27:13.198094 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.198071    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6fca471a-5cc7-4a75-aa8b-5be08533e7ea" (UID: "6fca471a-5cc7-4a75-aa8b-5be08533e7ea"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:27:13.198177 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.198146    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:27:13.199742 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.199718    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6fca471a-5cc7-4a75-aa8b-5be08533e7ea" (UID: "6fca471a-5cc7-4a75-aa8b-5be08533e7ea"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:27:13.199867 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.199767    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-api-access-xhfbf" (OuterVolumeSpecName: "kube-api-access-xhfbf") pod "6fca471a-5cc7-4a75-aa8b-5be08533e7ea" (UID: "6fca471a-5cc7-4a75-aa8b-5be08533e7ea"). InnerVolumeSpecName "kube-api-access-xhfbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:27:13.299034 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.299012    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:27:13.299034 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:13.299032    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhfbf\" (UniqueName: \"kubernetes.io/projected/6fca471a-5cc7-4a75-aa8b-5be08533e7ea-kube-api-access-xhfbf\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:27:14.081990 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:14.081963    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"
Jul 07 15:27:14.082378 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:14.081964    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8" event={"ID":"6fca471a-5cc7-4a75-aa8b-5be08533e7ea","Type":"ContainerDied","Data":"70fd7ba706ad038115fffe2b975e68cf5b38733fc3358edac4501adc50d638ad"}
Jul 07 15:27:14.082378 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:14.082109    2556 scope.go:117] "RemoveContainer" containerID="90106e952f7e2c34fcb62069dabc259f2df214db793ff63cfdf3dd77c4728c16"
Jul 07 15:27:14.090687 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:14.090670    2556 scope.go:117] "RemoveContainer" containerID="123f68cc82f6ad1b68fbc20b85b9d7ff18fec7243af5c603825934fd0eb71d19"
Jul 07 15:27:14.117052 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:14.117031    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"]
Jul 07 15:27:14.119658 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:14.119640    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9c06-predictor-bdb4cb5fd-dg4q8"]
Jul 07 15:27:15.269874 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:15.269841    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" path="/var/lib/kubelet/pods/6fca471a-5cc7-4a75-aa8b-5be08533e7ea/volumes"
Jul 07 15:27:18.079115 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:18.079084    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:27:18.079691 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:18.079658    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 07 15:27:21.899068 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:21.899039    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:27:28.079951 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:28.079913    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 07 15:27:38.080197 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:38.080162    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 07 15:27:44.849888 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.849853    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"]
Jul 07 15:27:44.850439 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.850419    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kube-rbac-proxy"
Jul 07 15:27:44.850538 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.850441    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kube-rbac-proxy"
Jul 07 15:27:44.850538 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.850500    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container"
Jul 07 15:27:44.850538 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.850509    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container"
Jul 07 15:27:44.850690 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.850596    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kube-rbac-proxy"
Jul 07 15:27:44.850690 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.850607    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fca471a-5cc7-4a75-aa8b-5be08533e7ea" containerName="kserve-container"
Jul 07 15:27:44.854008 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.853970    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:44.860623 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.860438    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-73e96-predictor-serving-cert\""
Jul 07 15:27:44.860623 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.860480    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-73e96-kube-rbac-proxy-sar-config\""
Jul 07 15:27:44.872673 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.872648    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"]
Jul 07 15:27:44.924443 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.924407    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:44.924611 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.924444    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54f7v\" (UniqueName: \"kubernetes.io/projected/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-api-access-54f7v\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:44.924611 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.924522    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:44.966860 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.966801    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"]
Jul 07 15:27:44.967066 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.967019    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container" containerID="cri-o://3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17" gracePeriod=30
Jul 07 15:27:44.967066 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:44.967050    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kube-rbac-proxy" containerID="cri-o://4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8" gracePeriod=30
Jul 07 15:27:45.025295 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.025272    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.025414 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.025302    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54f7v\" (UniqueName: \"kubernetes.io/projected/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-api-access-54f7v\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.025414 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.025354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.025542 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:27:45.025523    2556 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-73e96-predictor-serving-cert: secret "success-200-isvc-73e96-predictor-serving-cert" not found
Jul 07 15:27:45.025597 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:27:45.025586    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls podName:3fe24f63-5886-46fc-ab27-a003d32f4ef7 nodeName:}" failed. No retries permitted until 2026-07-07 15:27:45.525570513 +0000 UTC m=+1882.874053121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls") pod "success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" (UID: "3fe24f63-5886-46fc-ab27-a003d32f4ef7") : secret "success-200-isvc-73e96-predictor-serving-cert" not found
Jul 07 15:27:45.025920 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.025901    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.042780 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.042758    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54f7v\" (UniqueName: \"kubernetes.io/projected/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-api-access-54f7v\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.228471 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.228441    2556 generic.go:356] "Generic (PLEG): container finished" podID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerID="4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8" exitCode=2
Jul 07 15:27:45.228599 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.228493    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" event={"ID":"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9","Type":"ContainerDied","Data":"4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8"}
Jul 07 15:27:45.530031 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.529950    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.532303 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.532270    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls\") pod \"success-200-isvc-73e96-predictor-755bfdcbd-qkn2c\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") " pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.764732 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.764694    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:45.914008 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:45.913973    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"]
Jul 07 15:27:45.916724 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:27:45.916699    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe24f63_5886_46fc_ab27_a003d32f4ef7.slice/crio-e5d5704d492b3ab178613da9380d0c0c385a161e8923b4ee260d530b4626b3ff WatchSource:0}: Error finding container e5d5704d492b3ab178613da9380d0c0c385a161e8923b4ee260d530b4626b3ff: Status 404 returned error can't find the container with id e5d5704d492b3ab178613da9380d0c0c385a161e8923b4ee260d530b4626b3ff
Jul 07 15:27:46.235441 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.235408    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" event={"ID":"3fe24f63-5886-46fc-ab27-a003d32f4ef7","Type":"ContainerStarted","Data":"491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4"}
Jul 07 15:27:46.235640 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.235467    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:46.235640 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.235495    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" event={"ID":"3fe24f63-5886-46fc-ab27-a003d32f4ef7","Type":"ContainerStarted","Data":"761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0"}
Jul 07 15:27:46.235640 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.235605    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:46.235640 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.235627    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" event={"ID":"3fe24f63-5886-46fc-ab27-a003d32f4ef7","Type":"ContainerStarted","Data":"e5d5704d492b3ab178613da9380d0c0c385a161e8923b4ee260d530b4626b3ff"}
Jul 07 15:27:46.237042 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.237014    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 07 15:27:46.257034 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.256996    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podStartSLOduration=2.256983364 podStartE2EDuration="2.256983364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:27:46.255248242 +0000 UTC m=+1883.603730883" watchObservedRunningTime="2026-07-07 15:27:46.256983364 +0000 UTC m=+1883.605465992"
Jul 07 15:27:46.894379 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:46.894341    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.44:8643/healthz\": dial tcp 10.132.0.44:8643: connect: connection refused"
Jul 07 15:27:47.240109 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.240065    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 07 15:27:47.929814 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.929749    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:27:47.948858 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.948834    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb9s2\" (UniqueName: \"kubernetes.io/projected/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-api-access-fb9s2\") pod \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") "
Jul 07 15:27:47.948964 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.948873    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-proxy-tls\") pod \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") "
Jul 07 15:27:47.948964 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.948895    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-rbac-proxy-sar-config\") pod \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\" (UID: \"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9\") "
Jul 07 15:27:47.949264 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.949220    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" (UID: "e6ee6064-4c5d-411b-b0e5-57823a6a9fa9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:27:47.951438 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.951402    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" (UID: "e6ee6064-4c5d-411b-b0e5-57823a6a9fa9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:27:47.951538 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:47.951434    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-api-access-fb9s2" (OuterVolumeSpecName: "kube-api-access-fb9s2") pod "e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" (UID: "e6ee6064-4c5d-411b-b0e5-57823a6a9fa9"). InnerVolumeSpecName "kube-api-access-fb9s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:27:48.050099 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.050075    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb9s2\" (UniqueName: \"kubernetes.io/projected/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-api-access-fb9s2\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:27:48.050099 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.050097    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:27:48.050223 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.050108    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:27:48.080535 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.080507    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused"
Jul 07 15:27:48.246167 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.246087    2556 generic.go:356] "Generic (PLEG): container finished" podID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerID="3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17" exitCode=0
Jul 07 15:27:48.246167 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.246126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" event={"ID":"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9","Type":"ContainerDied","Data":"3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17"}
Jul 07 15:27:48.246593 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.246165    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"
Jul 07 15:27:48.246593 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.246193    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw" event={"ID":"e6ee6064-4c5d-411b-b0e5-57823a6a9fa9","Type":"ContainerDied","Data":"ecb4b3c759025f076cbe9d9128a11a319cf96d74101a0e751260d8c7d393d5c0"}
Jul 07 15:27:48.246593 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.246229    2556 scope.go:117] "RemoveContainer" containerID="4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8"
Jul 07 15:27:48.255501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.255483    2556 scope.go:117] "RemoveContainer" containerID="3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17"
Jul 07 15:27:48.263437 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.263421    2556 scope.go:117] "RemoveContainer" containerID="4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8"
Jul 07 15:27:48.263693 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:27:48.263675    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8\": container with ID starting with 4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8 not found: ID does not exist" containerID="4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8"
Jul 07 15:27:48.263749 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.263702    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8"} err="failed to get container status \"4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8\": rpc error: code = NotFound desc = could not find container \"4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8\": container with ID starting with 4f593c35bbf76730c218bbb82ab86a211ec78fbbc1980c3743a3fb856b5bd6e8 not found: ID does not exist"
Jul 07 15:27:48.263749 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.263718    2556 scope.go:117] "RemoveContainer" containerID="3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17"
Jul 07 15:27:48.263967 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:27:48.263951    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17\": container with ID starting with 3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17 not found: ID does not exist" containerID="3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17"
Jul 07 15:27:48.264011 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.263972    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17"} err="failed to get container status \"3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17\": rpc error: code = NotFound desc = could not find container \"3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17\": container with ID starting with 3375c41013187e6e5c540d0700b2df3db04c2b4abe116db11859688ccf427a17 not found: ID does not exist"
Jul 07 15:27:48.273847 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.273818    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"]
Jul 07 15:27:48.285395 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:48.285374    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-761e8-predictor-6d97467478-f9qnw"]
Jul 07 15:27:49.269648 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:49.269619    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" path="/var/lib/kubelet/pods/e6ee6064-4c5d-411b-b0e5-57823a6a9fa9/volumes"
Jul 07 15:27:52.245141 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:52.245115    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:27:52.245621 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:52.245590    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 07 15:27:58.080368 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:27:58.080342    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:28:02.245824 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:28:02.245752    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 07 15:28:12.245961 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:28:12.245913    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 07 15:28:22.246325 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:28:22.246286    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 07 15:28:32.246417 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:28:32.246386    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:31:23.305095 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:31:23.305064    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:31:23.307450 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:31:23.306232    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:31:23.310405 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:31:23.310388    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:31:23.311664 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:31:23.311646    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:36:23.334391 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:23.334289    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:36:23.338287 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:23.336323    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:36:23.340598 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:23.340568    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:36:23.342175 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:23.342154    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:36:59.694060 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:59.694023    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"]
Jul 07 15:36:59.694663 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:59.694314    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" containerID="cri-o://761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0" gracePeriod=30
Jul 07 15:36:59.694663 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:59.694356    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kube-rbac-proxy" containerID="cri-o://491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4" gracePeriod=30
Jul 07 15:36:59.838567 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:59.838537    2556 generic.go:356] "Generic (PLEG): container finished" podID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerID="491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4" exitCode=2
Jul 07 15:36:59.838679 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:36:59.838610    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" event={"ID":"3fe24f63-5886-46fc-ab27-a003d32f4ef7","Type":"ContainerDied","Data":"491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4"}
Jul 07 15:37:02.241211 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.241173    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.46:8643/healthz\": dial tcp 10.132.0.46:8643: connect: connection refused"
Jul 07 15:37:02.245644 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.245611    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused"
Jul 07 15:37:02.346670 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.346648    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:37:02.440093 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.440021    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-rbac-proxy-sar-config\") pod \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") "
Jul 07 15:37:02.440093 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.440057    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54f7v\" (UniqueName: \"kubernetes.io/projected/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-api-access-54f7v\") pod \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") "
Jul 07 15:37:02.440093 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.440078    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls\") pod \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\" (UID: \"3fe24f63-5886-46fc-ab27-a003d32f4ef7\") "
Jul 07 15:37:02.440401 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.440377    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3fe24f63-5886-46fc-ab27-a003d32f4ef7" (UID: "3fe24f63-5886-46fc-ab27-a003d32f4ef7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:37:02.442132 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.442103    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3fe24f63-5886-46fc-ab27-a003d32f4ef7" (UID: "3fe24f63-5886-46fc-ab27-a003d32f4ef7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:37:02.442243 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.442178    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-api-access-54f7v" (OuterVolumeSpecName: "kube-api-access-54f7v") pod "3fe24f63-5886-46fc-ab27-a003d32f4ef7" (UID: "3fe24f63-5886-46fc-ab27-a003d32f4ef7"). InnerVolumeSpecName "kube-api-access-54f7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:37:02.540600 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.540573    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:37:02.540600 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.540597    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54f7v\" (UniqueName: \"kubernetes.io/projected/3fe24f63-5886-46fc-ab27-a003d32f4ef7-kube-api-access-54f7v\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:37:02.540770 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.540606    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fe24f63-5886-46fc-ab27-a003d32f4ef7-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:37:02.853206 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.853124    2556 generic.go:356] "Generic (PLEG): container finished" podID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerID="761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0" exitCode=0
Jul 07 15:37:02.853206 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.853174    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"
Jul 07 15:37:02.853206 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.853187    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" event={"ID":"3fe24f63-5886-46fc-ab27-a003d32f4ef7","Type":"ContainerDied","Data":"761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0"}
Jul 07 15:37:02.853440 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.853252    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c" event={"ID":"3fe24f63-5886-46fc-ab27-a003d32f4ef7","Type":"ContainerDied","Data":"e5d5704d492b3ab178613da9380d0c0c385a161e8923b4ee260d530b4626b3ff"}
Jul 07 15:37:02.853440 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.853293    2556 scope.go:117] "RemoveContainer" containerID="491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4"
Jul 07 15:37:02.862710 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.862691    2556 scope.go:117] "RemoveContainer" containerID="761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0"
Jul 07 15:37:02.869918 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.869900    2556 scope.go:117] "RemoveContainer" containerID="491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4"
Jul 07 15:37:02.870149 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:37:02.870131    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4\": container with ID starting with 491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4 not found: ID does not exist" containerID="491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4"
Jul 07 15:37:02.870194 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.870158    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4"} err="failed to get container status \"491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4\": rpc error: code = NotFound desc = could not find container \"491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4\": container with ID starting with 491445560ea0e060172b5811ff063b8f17c67e0281b8d452ec432b303c4dbeb4 not found: ID does not exist"
Jul 07 15:37:02.870194 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.870174    2556 scope.go:117] "RemoveContainer" containerID="761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0"
Jul 07 15:37:02.870418 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:37:02.870398    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0\": container with ID starting with 761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0 not found: ID does not exist" containerID="761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0"
Jul 07 15:37:02.870472 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.870424    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0"} err="failed to get container status \"761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0\": rpc error: code = NotFound desc = could not find container \"761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0\": container with ID starting with 761e439a21fe472e5d44c63b1beba696a4e455dcb489d6f699427a5aef9816c0 not found: ID does not exist"
Jul 07 15:37:02.876308 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.876286    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"]
Jul 07 15:37:02.882433 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:02.882414    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-73e96-predictor-755bfdcbd-qkn2c"]
Jul 07 15:37:03.269973 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:37:03.269949    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" path="/var/lib/kubelet/pods/3fe24f63-5886-46fc-ab27-a003d32f4ef7/volumes"
Jul 07 15:41:23.366899 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:41:23.366777    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:41:23.370901 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:41:23.369915    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:41:23.372718 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:41:23.372698    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:41:23.375679 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:41:23.375660    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:44:29.555228 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:29.555125    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"]
Jul 07 15:44:29.555820 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:29.555449    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container" containerID="cri-o://de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8" gracePeriod=30
Jul 07 15:44:29.555820 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:29.555492    2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kube-rbac-proxy" containerID="cri-o://3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7" gracePeriod=30
Jul 07 15:44:29.905443 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:29.905361    2556 generic.go:356] "Generic (PLEG): container finished" podID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerID="3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7" exitCode=2
Jul 07 15:44:29.905443 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:29.905433    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" event={"ID":"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f","Type":"ContainerDied","Data":"3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7"}
Jul 07 15:44:32.206394 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.206372    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:44:32.233638 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.233576    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-proxy-tls\") pod \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") "
Jul 07 15:44:32.233764 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.233664    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4m79\" (UniqueName: \"kubernetes.io/projected/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-api-access-j4m79\") pod \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") "
Jul 07 15:44:32.233764 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.233688    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-rbac-proxy-sar-config\") pod \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\" (UID: \"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f\") "
Jul 07 15:44:32.234070 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.234044    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" (UID: "bac4cca2-eea1-4e92-a48c-d8ba22d11f0f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:44:32.235582 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.235558    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" (UID: "bac4cca2-eea1-4e92-a48c-d8ba22d11f0f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:44:32.235670 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.235656    2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-api-access-j4m79" (OuterVolumeSpecName: "kube-api-access-j4m79") pod "bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" (UID: "bac4cca2-eea1-4e92-a48c-d8ba22d11f0f"). InnerVolumeSpecName "kube-api-access-j4m79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:44:32.335117 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.335094    2556 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-proxy-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:44:32.335117 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.335113    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4m79\" (UniqueName: \"kubernetes.io/projected/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-api-access-j4m79\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:44:32.335248 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.335123    2556 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 07 15:44:32.921369 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.921338    2556 generic.go:356] "Generic (PLEG): container finished" podID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerID="de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8" exitCode=0
Jul 07 15:44:32.921369 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.921378    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" event={"ID":"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f","Type":"ContainerDied","Data":"de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8"}
Jul 07 15:44:32.921630 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.921384    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"
Jul 07 15:44:32.921630 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.921434    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj" event={"ID":"bac4cca2-eea1-4e92-a48c-d8ba22d11f0f","Type":"ContainerDied","Data":"4e02174a32ff0727c75c5be57ed02efbdc446af03ba7ad6313b44fd4d5d1b1af"}
Jul 07 15:44:32.921630 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.921472    2556 scope.go:117] "RemoveContainer" containerID="3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7"
Jul 07 15:44:32.930296 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.930280    2556 scope.go:117] "RemoveContainer" containerID="de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8"
Jul 07 15:44:32.937490 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.937472    2556 scope.go:117] "RemoveContainer" containerID="3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7"
Jul 07 15:44:32.937728 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:44:32.937711    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7\": container with ID starting with 3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7 not found: ID does not exist" containerID="3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7"
Jul 07 15:44:32.937774 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.937737    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7"} err="failed to get container status \"3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7\": rpc error: code = NotFound desc = could not find container \"3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7\": container with ID starting with 3789f5f82ca247ceceff56e82a1f60bb8515fc65fd5302bee8c8340a2e7dc8c7 not found: ID does not exist"
Jul 07 15:44:32.937774 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.937753    2556 scope.go:117] "RemoveContainer" containerID="de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8"
Jul 07 15:44:32.937986 ip-10-0-134-178 kubenswrapper[2556]: E0707 15:44:32.937971    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8\": container with ID starting with de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8 not found: ID does not exist" containerID="de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8"
Jul 07 15:44:32.938034 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.937989    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8"} err="failed to get container status \"de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8\": rpc error: code = NotFound desc = could not find container \"de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8\": container with ID starting with de804aeb0a7f7d2103e16e339f73aa198500762d215cd7ada34f4d7ec85060c8 not found: ID does not exist"
Jul 07 15:44:32.943376 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.943356    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"]
Jul 07 15:44:32.949811 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:32.949454    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e9b-predictor-8557bd9dd4-9r7lj"]
Jul 07 15:44:33.270083 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:44:33.270012    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" path="/var/lib/kubelet/pods/bac4cca2-eea1-4e92-a48c-d8ba22d11f0f/volumes"
Jul 07 15:45:01.569020 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.568979    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4xx6g/must-gather-g564j"]
Jul 07 15:45:01.569501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569418    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container"
Jul 07 15:45:01.569501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569437    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container"
Jul 07 15:45:01.569501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569451    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569460    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569483    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569501 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569493    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569522    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569532    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569541    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569550    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569563    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569571    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569655    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kserve-container"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569666    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569678    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569689    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe24f63-5886-46fc-ab27-a003d32f4ef7" containerName="kube-rbac-proxy"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569697    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6ee6064-4c5d-411b-b0e5-57823a6a9fa9" containerName="kserve-container"
Jul 07 15:45:01.569706 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.569704    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac4cca2-eea1-4e92-a48c-d8ba22d11f0f" containerName="kserve-container"
Jul 07 15:45:01.574173 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.574150    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:01.578411 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.578390    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4xx6g\"/\"kube-root-ca.crt\""
Jul 07 15:45:01.579284 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.579261    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4xx6g\"/\"default-dockercfg-s4f8t\""
Jul 07 15:45:01.579405 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.579267    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4xx6g\"/\"openshift-service-ca.crt\""
Jul 07 15:45:01.582673 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.582654    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4xx6g/must-gather-g564j"]
Jul 07 15:45:01.664095 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.664065    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d268a2c5-9a75-44bb-867f-3ea1faf28957-must-gather-output\") pod \"must-gather-g564j\" (UID: \"d268a2c5-9a75-44bb-867f-3ea1faf28957\") " pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:01.664219 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.664099    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv8vw\" (UniqueName: \"kubernetes.io/projected/d268a2c5-9a75-44bb-867f-3ea1faf28957-kube-api-access-zv8vw\") pod \"must-gather-g564j\" (UID: \"d268a2c5-9a75-44bb-867f-3ea1faf28957\") " pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:01.765196 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.765172    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d268a2c5-9a75-44bb-867f-3ea1faf28957-must-gather-output\") pod \"must-gather-g564j\" (UID: \"d268a2c5-9a75-44bb-867f-3ea1faf28957\") " pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:01.765292 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.765204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv8vw\" (UniqueName: \"kubernetes.io/projected/d268a2c5-9a75-44bb-867f-3ea1faf28957-kube-api-access-zv8vw\") pod \"must-gather-g564j\" (UID: \"d268a2c5-9a75-44bb-867f-3ea1faf28957\") " pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:01.765493 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.765476    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d268a2c5-9a75-44bb-867f-3ea1faf28957-must-gather-output\") pod \"must-gather-g564j\" (UID: \"d268a2c5-9a75-44bb-867f-3ea1faf28957\") " pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:01.773111 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.773087    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv8vw\" (UniqueName: \"kubernetes.io/projected/d268a2c5-9a75-44bb-867f-3ea1faf28957-kube-api-access-zv8vw\") pod \"must-gather-g564j\" (UID: \"d268a2c5-9a75-44bb-867f-3ea1faf28957\") " pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:01.889682 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:01.889629    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4xx6g/must-gather-g564j"
Jul 07 15:45:02.017818 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:02.017781    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4xx6g/must-gather-g564j"]
Jul 07 15:45:02.020395 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:45:02.020368    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd268a2c5_9a75_44bb_867f_3ea1faf28957.slice/crio-48386f2a450d2f65200f795ae80e3b9b8cedede21070ed3b4404b1f8fd8855dd WatchSource:0}: Error finding container 48386f2a450d2f65200f795ae80e3b9b8cedede21070ed3b4404b1f8fd8855dd: Status 404 returned error can't find the container with id 48386f2a450d2f65200f795ae80e3b9b8cedede21070ed3b4404b1f8fd8855dd
Jul 07 15:45:02.022181 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:02.022164    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:45:02.047559 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:02.047529    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4xx6g/must-gather-g564j" event={"ID":"d268a2c5-9a75-44bb-867f-3ea1faf28957","Type":"ContainerStarted","Data":"48386f2a450d2f65200f795ae80e3b9b8cedede21070ed3b4404b1f8fd8855dd"}
Jul 07 15:45:04.061109 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:04.061074    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4xx6g/must-gather-g564j" event={"ID":"d268a2c5-9a75-44bb-867f-3ea1faf28957","Type":"ContainerStarted","Data":"185660c02cc677ce5cec3cd563f12c86380471f30c8553620c2278b508ce7681"}
Jul 07 15:45:04.061500 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:04.061121    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4xx6g/must-gather-g564j" event={"ID":"d268a2c5-9a75-44bb-867f-3ea1faf28957","Type":"ContainerStarted","Data":"151fe668e1eef1fc36f3befe99d53b667eaaa7568b33a840a7983c8a8e64de91"}
Jul 07 15:45:04.081098 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:04.081041    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4xx6g/must-gather-g564j" podStartSLOduration=1.835282562 podStartE2EDuration="3.081020649s" totalImagesPullingTime="1.245738087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:04.078882927 +0000 UTC m=+2921.427365556" watchObservedRunningTime="2026-07-07 15:45:04.081020649 +0000 UTC m=+2921.429503282"
Jul 07 15:45:04.773818 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:04.773762    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f7mm6_dec33f92-47f6-4040-8a3b-4658fdbf72ca/global-pull-secret-syncer/0.log"
Jul 07 15:45:04.944162 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:04.944131    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bpb5c_c835e94b-6053-4750-b83d-354e135e1e8f/konnectivity-agent/0.log"
Jul 07 15:45:05.103671 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:05.103585    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-178.ec2.internal_90b7955341960021bd0b5ed66f077c6f/haproxy/0.log"
Jul 07 15:45:08.952998 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:08.952967    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23ff7c29-eab4-46c8-831f-8aa51b684134/alertmanager/0.log"
Jul 07 15:45:08.985891 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:08.985862    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23ff7c29-eab4-46c8-831f-8aa51b684134/config-reloader/0.log"
Jul 07 15:45:09.015185 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.015151    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23ff7c29-eab4-46c8-831f-8aa51b684134/kube-rbac-proxy-web/0.log"
Jul 07 15:45:09.041163 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.041129    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23ff7c29-eab4-46c8-831f-8aa51b684134/kube-rbac-proxy/0.log"
Jul 07 15:45:09.070471 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.070439    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23ff7c29-eab4-46c8-831f-8aa51b684134/kube-rbac-proxy-metric/0.log"
Jul 07 15:45:09.108214 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.108172    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23ff7c29-eab4-46c8-831f-8aa51b684134/prom-label-proxy/0.log"
Jul 07 15:45:09.139943 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.139916    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23ff7c29-eab4-46c8-831f-8aa51b684134/init-config-reloader/0.log"
Jul 07 15:45:09.204488 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.204405    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-pvr7j_0227a3b2-bc6c-4274-91f5-5dee553832d3/cluster-monitoring-operator/0.log"
Jul 07 15:45:09.357641 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.357608    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-fdbf56d4b-zzlp8_707b7933-24bd-4d4c-b895-9530f2477cbd/metrics-server/0.log"
Jul 07 15:45:09.521938 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.521867    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77t6k_e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e/node-exporter/0.log"
Jul 07 15:45:09.546250 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.546223    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77t6k_e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e/kube-rbac-proxy/0.log"
Jul 07 15:45:09.569728 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:09.569698    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77t6k_e31f5eb6-f858-4da0-98d4-7bf4ccb48d7e/init-textfile/0.log"
Jul 07 15:45:10.000834 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.000781    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-knkrb_1aced028-98bf-45c7-a2e6-69e6476d4b0d/prometheus-operator-admission-webhook/0.log"
Jul 07 15:45:10.041779 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.041747    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c7f6c5c9b-8b5xg_059c013a-575c-4397-9d5b-90770213f152/telemeter-client/0.log"
Jul 07 15:45:10.075954 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.075925    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c7f6c5c9b-8b5xg_059c013a-575c-4397-9d5b-90770213f152/reload/0.log"
Jul 07 15:45:10.100479 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.100445    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c7f6c5c9b-8b5xg_059c013a-575c-4397-9d5b-90770213f152/kube-rbac-proxy/0.log"
Jul 07 15:45:10.138591 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.138559    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-566f8c8df6-kc4b2_3e04a0fc-df5e-4d89-9a4b-e252354230a4/thanos-query/0.log"
Jul 07 15:45:10.171274 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.171245    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-566f8c8df6-kc4b2_3e04a0fc-df5e-4d89-9a4b-e252354230a4/kube-rbac-proxy-web/0.log"
Jul 07 15:45:10.200217 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.200185    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-566f8c8df6-kc4b2_3e04a0fc-df5e-4d89-9a4b-e252354230a4/kube-rbac-proxy/0.log"
Jul 07 15:45:10.224921 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.224864    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-566f8c8df6-kc4b2_3e04a0fc-df5e-4d89-9a4b-e252354230a4/prom-label-proxy/0.log"
Jul 07 15:45:10.252349 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.252267    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-566f8c8df6-kc4b2_3e04a0fc-df5e-4d89-9a4b-e252354230a4/kube-rbac-proxy-rules/0.log"
Jul 07 15:45:10.280115 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:10.280083    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-566f8c8df6-kc4b2_3e04a0fc-df5e-4d89-9a4b-e252354230a4/kube-rbac-proxy-metrics/0.log"
Jul 07 15:45:11.517394 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:11.517363    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-qnkcv_7b518f69-69ee-4d12-b33e-d4a7d4792b55/networking-console-plugin/0.log"
Jul 07 15:45:12.033637 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:12.033582    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/1.log"
Jul 07 15:45:12.044040 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:12.044014    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-k8r9l_77040db7-a4f5-4195-8504-062e41f25c55/console-operator/2.log"
Jul 07 15:45:12.461941 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:12.461905    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f49b9ccbf-nc5tf_f69eedeb-977f-4072-bd6e-91cbf308f9c8/console/0.log"
Jul 07 15:45:12.507006 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:12.506970    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-php5t_b1c843ae-67f8-4d36-80cc-6fb63018755a/download-server/0.log"
Jul 07 15:45:12.945634 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:12.945589    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"]
Jul 07 15:45:12.952444 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:12.952412    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:12.959081 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:12.959051    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"]
Jul 07 15:45:13.057760 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.057722    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-8p572_02ccf314-fc77-4a3e-b795-883a2a897ba5/volume-data-source-validator/0.log"
Jul 07 15:45:13.074709 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.074676    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-proc\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.074912 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.074747    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-lib-modules\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.074912 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.074860    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-podres\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.075043 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.074917    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87kk2\" (UniqueName: \"kubernetes.io/projected/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-kube-api-access-87kk2\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.075043 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.074996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-sys\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.175819 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.175765    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-proc\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.175819 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.175813    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-lib-modules\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.176079 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.175860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-podres\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.176079 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.175886    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-proc\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.176079 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.175892    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87kk2\" (UniqueName: \"kubernetes.io/projected/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-kube-api-access-87kk2\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.176079 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.175965    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-sys\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.176079 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.175982    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-podres\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.176079 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.176004    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-lib-modules\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.176079 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.176069    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-sys\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.185313 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.185286    2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87kk2\" (UniqueName: \"kubernetes.io/projected/0316f48e-6d0f-4a16-af0e-b25ea82fc8b8-kube-api-access-87kk2\") pod \"perf-node-gather-daemonset-wh8qq\" (UID: \"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.264453 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.264373    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:13.438550 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:13.437509    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"]
Jul 07 15:45:13.440717 ip-10-0-134-178 kubenswrapper[2556]: W0707 15:45:13.440666    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0316f48e_6d0f_4a16_af0e_b25ea82fc8b8.slice/crio-64d4704e9a3764b9f2f72e6e2db88693a5ecf507e39836930fac9edb74821b07 WatchSource:0}: Error finding container 64d4704e9a3764b9f2f72e6e2db88693a5ecf507e39836930fac9edb74821b07: Status 404 returned error can't find the container with id 64d4704e9a3764b9f2f72e6e2db88693a5ecf507e39836930fac9edb74821b07
Jul 07 15:45:14.094502 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.094473    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xjl76_0718570a-e9fe-460b-a0c9-b895125c2d44/dns/0.log"
Jul 07 15:45:14.118507 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.118481    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xjl76_0718570a-e9fe-460b-a0c9-b895125c2d44/kube-rbac-proxy/0.log"
Jul 07 15:45:14.123578 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.123544    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq" event={"ID":"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8","Type":"ContainerStarted","Data":"9ed8fdfdfc94cb3c989ea6c8179da12844c823e996b9f05d8033e4553d6b82cc"}
Jul 07 15:45:14.123752 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.123606    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq" event={"ID":"0316f48e-6d0f-4a16-af0e-b25ea82fc8b8","Type":"ContainerStarted","Data":"64d4704e9a3764b9f2f72e6e2db88693a5ecf507e39836930fac9edb74821b07"}
Jul 07 15:45:14.123752 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.123664    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:14.141965 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.141918    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq" podStartSLOduration=2.141902423 podStartE2EDuration="2.141902423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:14.139266701 +0000 UTC m=+2931.487749357" watchObservedRunningTime="2026-07-07 15:45:14.141902423 +0000 UTC m=+2931.490385067"
Jul 07 15:45:14.200105 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.200082    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x4xx9_0b19a2c4-299a-4d9e-a470-50b1d292899f/dns-node-resolver/0.log"
Jul 07 15:45:14.720188 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:14.720157    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9q9q5_91074283-7c9c-4a20-9196-2d55a7bc6ae1/node-ca/0.log"
Jul 07 15:45:15.641864 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:15.641832    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f8f66bc9c-fhp6v_272f03a0-8b74-47c6-b8f5-7d82a5d4bba8/router/0.log"
Jul 07 15:45:16.043305 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:16.043275    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dht8q_c0c5e587-0e58-4aa8-b704-350d87c04978/serve-healthcheck-canary/0.log"
Jul 07 15:45:16.441538 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:16.441509    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-n5ptf_c2992c03-9dce-432d-a63e-99a5c3723d06/insights-operator/0.log"
Jul 07 15:45:16.442935 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:16.442911    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-n5ptf_c2992c03-9dce-432d-a63e-99a5c3723d06/insights-operator/1.log"
Jul 07 15:45:16.466519 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:16.466498    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mb4h_71190e89-69d2-4d28-8825-f3de21d2de28/kube-rbac-proxy/0.log"
Jul 07 15:45:16.491580 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:16.491557    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mb4h_71190e89-69d2-4d28-8825-f3de21d2de28/exporter/0.log"
Jul 07 15:45:16.516328 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:16.516309    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mb4h_71190e89-69d2-4d28-8825-f3de21d2de28/extractor/0.log"
Jul 07 15:45:18.941637 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:18.941600    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-666b75c8f7-9q6c4_b17b94b1-9a20-46ea-a1dc-75d2d01b4abb/manager/0.log"
Jul 07 15:45:19.149591 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:19.149554    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-lc964_70ce05a7-b0f4-47cd-b017-c872f8054b2b/manager/0.log"
Jul 07 15:45:19.246018 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:19.245956    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-6vmr2_924ebc4e-458f-47d3-8d66-4dd15d9e3103/seaweedfs/0.log"
Jul 07 15:45:20.141631 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:20.141601    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-wh8qq"
Jul 07 15:45:25.131321 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.131218    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6gbg_c0fc80cb-6c79-482c-b8f3-d6856ea7d72c/kube-multus-additional-cni-plugins/0.log"
Jul 07 15:45:25.155620 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.155592    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6gbg_c0fc80cb-6c79-482c-b8f3-d6856ea7d72c/egress-router-binary-copy/0.log"
Jul 07 15:45:25.177932 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.177911    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6gbg_c0fc80cb-6c79-482c-b8f3-d6856ea7d72c/cni-plugins/0.log"
Jul 07 15:45:25.201142 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.201118    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6gbg_c0fc80cb-6c79-482c-b8f3-d6856ea7d72c/bond-cni-plugin/0.log"
Jul 07 15:45:25.221976 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.221956    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6gbg_c0fc80cb-6c79-482c-b8f3-d6856ea7d72c/routeoverride-cni/0.log"
Jul 07 15:45:25.243865 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.243842    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6gbg_c0fc80cb-6c79-482c-b8f3-d6856ea7d72c/whereabouts-cni-bincopy/0.log"
Jul 07 15:45:25.264655 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.264635    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6gbg_c0fc80cb-6c79-482c-b8f3-d6856ea7d72c/whereabouts-cni/0.log"
Jul 07 15:45:25.306184 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.306139    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nf625_7246a8b1-2ce5-412f-925b-27e83323ed25/kube-multus/0.log"
Jul 07 15:45:25.425270 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.425232    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wxd9k_b6fd483f-453e-4289-82fb-02ad4d241017/network-metrics-daemon/0.log"
Jul 07 15:45:25.449071 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:25.449034    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wxd9k_b6fd483f-453e-4289-82fb-02ad4d241017/kube-rbac-proxy/0.log"
Jul 07 15:45:26.680286 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.680215    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-controller/0.log"
Jul 07 15:45:26.701919 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.701890    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/0.log"
Jul 07 15:45:26.701919 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.701926    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovn-acl-logging/1.log"
Jul 07 15:45:26.727033 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.726996    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/kube-rbac-proxy-node/0.log"
Jul 07 15:45:26.753003 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.752979    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/kube-rbac-proxy-ovn-metrics/0.log"
Jul 07 15:45:26.778197 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.778167    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/northd/0.log"
Jul 07 15:45:26.803840 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.803819    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/nbdb/0.log"
Jul 07 15:45:26.832733 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.832711    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/sbdb/0.log"
Jul 07 15:45:26.904625 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:26.904594    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ssnrw_d444a611-792d-4ff0-9cca-cda0856d9577/ovnkube-controller/0.log"
Jul 07 15:45:27.974752 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:27.974717    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-756dp_70153084-d026-4d8c-a91c-dff91ff1f515/network-check-target-container/0.log"
Jul 07 15:45:29.068629 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:29.068605    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-d9rmr_cb5585b7-b224-4b7b-b61b-ccdd7c3165b9/iptables-alerter/0.log"
Jul 07 15:45:29.807739 ip-10-0-134-178 kubenswrapper[2556]: I0707 15:45:29.807703    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-x5xvg_8dfa10f2-79ee-4bca-8ba6-65fdce870661/tuned/0.log"